CN104333488B - Cloud service platform performance test methods - Google Patents
Cloud service platform performance test methods Download PDFInfo
- Publication number
- CN104333488B CN104333488B CN201410614010.4A CN201410614010A CN104333488B CN 104333488 B CN104333488 B CN 104333488B CN 201410614010 A CN201410614010 A CN 201410614010A CN 104333488 B CN104333488 B CN 104333488B
- Authority
- CN
- China
- Prior art keywords
- test
- performance
- cloud service
- virtual machine
- service platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
Cloud service platform performance test methods, belong to computer cloud service technical field, are related to a kind of cloud service platform performance test methods, and the present invention is to solve the problems, such as that existing testing tool and test event cannot meet cloud service platform performance evaluating.Performance test methods of the invention are tested the service interface of cloud service platform layer, platform management layer and virtual resource layer；The method of testing includes that virtualizing performance test, resource management capacity test and service provide aptitude tests；Virtualizing performance tests the performance of the virtual resource for testing cloud service platform bottom；Resource management capacity tests the managerial ability for testing cloud service platform to virtual resource；Service provision capacity is tested for testing the ability that cloud service platform provides external user service.The present invention to cloud service platform for evaluating and testing.
Description
Technical field
The invention belongs to computer cloud service technical field, it is related to a kind of cloud service platform performance test methods.
Background technology
Since the new century, information technology and IT industry development it is very rapid, traditional computation schema has been difficult
Deacclimatize the demand of ever-increasing data processing, it is therefore desirable to a kind of new, efficient computation schema.Cloud computing can be carried
It is that big data epoch effective data calculate solution for general efficient computing platforms and memory space, and can be big
Amplitude is cost-effective.
Cloud computing technology has broken the limitation of traditional server, and user only needs to a terminal device and network connection, just
The software resource of the physical resources such as processor resource, storage resource and various operations environment beyond the clouds can be obtained, but also can
Taken on demand with telescopic flexible resource pattern with as obtaining coal gas, water power.According to the definition of NTSI, with reference to cloud computing
Cloud computing can be divided into three kinds of service modes by the service of provider and Consumer's Experience angle：IaaS provide include storage, virtually
The hardware services such as server；PaaS is typically based on IaaS and provides service than IaaS more top, such as Software Development Platform；SaaS
Provide it is a kind of it is specific, via the software service of internet.At present on the ununified definition of cloud service platform, each
Cloud service supplier is the business according to oneself enterprise releases the cloud service platform scheme of correlation.Sensu lato cloud service platform
Should include from the hardware enclosure of bottom to the application service of top layer, as shown in the cloud service platform framework map of Fig. 1.
But, developing rapidly and popularizing with cloud computing also brings many new problems, wherein to cloud service platform
The evaluation and test of aspect of performance be wherein vital one side.Due to compared with traditional calculations platform, cloud service platform
Complexity is higher, and scale is also huger, therefore is a highly difficult task to the performance test of cloud service platform.
The content of the invention
Cannot meet cloud service platform performance and comment the invention aims to solve existing testing tool and test event
A kind of problem of survey, there is provided cloud service platform performance test methods.
Cloud service platform performance test methods of the present invention, the cloud service platform includes service interface layer, platform pipe
Reason layer, virtual resource layer and physical resource layer, physical resource layer are the physical hardware of bottom, and virtual resource layer is used for bottom
Physical resource is virtualized, and platform management layer is used for cloud for scheduling virtual machine, monitoring performance and management resource, service interface layer
The computing capability of service platform is packaged into service interface and is supplied to external user；
The cloud service platform performance test methods, the performance test methods are to the service interface of cloud service platform layer, flat
Platform management level and virtual resource layer are tested；The method of testing include virtualizing performance test, resource management capacity test and
Service provision capacity is tested；
Virtualizing performance tests the performance of the virtual resource for testing cloud service platform bottom；
Resource management capacity tests the managerial ability for testing cloud service platform to virtual resource；
Service provision capacity is tested for testing the ability that cloud service platform provides external user service.
Virtualizing performance test detailed process be：
Step one, cloud service platform is logged in, virtual machine is asked to calculate node by control node；
Step 2, calculate node start virtual machine according to the parameter configuration of request；
Step 3, according to the configuration of test event and virtual machine set test parameter；
Step 4, the order of testing tool and the order of testing results instrument are packaged into an automatic test script, and
Test script is uploaded on virtual machine；
Step 5, on a virtual machine testing results script are tested, and obtain test index.
Resource management capacity test detailed process be：
Step one, cloud service platform send request according to test event to management node；
Step 2, management node are tested according to the corresponding test module of request call, while empty to cloud service platform
Plan machine carries out corresponding configuration；
Step 3, acquisition test index, return to management node.
Advantages of the present invention：The present invention combines cloud service platform framework, cloud service platform method of testing is proposed, from three layers
It is secondary that performance test is carried out to cloud service platform, virtualizing performance, resource management capacity and the service of cloud service platform are tested respectively
Offer ability, corresponds to virtual resource layer, platform management layer and the application service interface layer of cloud service platform, in each level respectively
It is respectively provided with multiple test events and corresponding test index.In the test process of each test event, while usability
The performance monitoring function of energy monitoring tools and cloud service platform is monitored to the property indices of testing service device, obtains each
Plant performance parameter.The overall hierarchy of cloud service platform performance test methods is as shown in Figure 2.
Brief description of the drawings
Fig. 1 is the framework map of cloud service platform；
Fig. 2 is the overall hierarchy schematic diagram of cloud service platform performance test methods；
Fig. 3 be in cloud service platform performance test methods service provision capacity test based on LoadRunner load testings
Tool Framework FIGURE.
Specific embodiment
Specific embodiment one：Present embodiment, cloud service platform performance described in present embodiment are illustrated with reference to Fig. 2
Method of testing, the cloud service platform includes service interface layer, platform management layer, virtual resource layer and physical resource layer, physics
Resource layer is the physical hardware of bottom, and, for the physical resource of bottom to be virtualized, platform management layer is for adjusting for virtual resource layer
Degree virtual machine, monitoring performance and management resource, service interface layer connect for the computing capability of cloud service platform to be packaged into service
Mouth is supplied to external user；
The cloud service platform performance test methods, the performance test methods are to the service interface of cloud service platform layer, flat
Platform management level and virtual resource layer are tested；The method of testing include virtualizing performance test, resource management capacity test and
Service provision capacity is tested；
Virtualizing performance tests the performance of the virtual resource for testing cloud service platform bottom；
Resource management capacity tests the managerial ability for testing cloud service platform to virtual resource；
Service provision capacity is tested for testing the ability that cloud service platform provides external user service.
In present embodiment, physical resource layer represents the physical hardware of bottom, including computer, memory and the network facilities
Deng；Virtual resource layer can rapidly, resource is dynamically provided；The responsible scheduling virtual machine of platform management layer, performance monitoring and money
Source control works, and manages, dispatches the virtual resource of lower floor and for the application on upper strata provides support；Service interface layer puts down cloud service
The computing capability of platform is packaged into service interface, there is provided used to outside user.
Specific embodiment two：Present embodiment is described further to implementation method one, the tool of virtualizing performance test
Body process is：
Step one, cloud service platform is logged in, virtual machine is asked to calculate node by control node；
Step 2, calculate node start virtual machine according to the parameter configuration of request；
Step 3, according to the configuration of test event and virtual machine set test parameter；
Step 4, the order of testing tool and the order of testing results instrument are packaged into an automatic test script, and
Test script is uploaded on virtual machine；
Step 5, on a virtual machine testing results script are tested, and obtain test index.
Specific embodiment three：Present embodiment is described further to implementation method two, the virtualizing performance test
Test event include virtual machine performance test, cluster virtual machine performance test and virtualizing performance loss test；
The test content of virtual machine performance test includes processor performance test, applied in network performance test, internal memory performance test
Tested with disk performance；
The test index of processor performance test is computation-intensive task process time；The test of applied in network performance test refers to
It is designated as network delay and packet loss；The test index of internal memory performance test is memory bandwidth；The test index of disk performance test
It is disk read-write speed.
In present embodiment, cloud computing is at all virtualization, in cloud service platform, by Intel Virtualization Technology physics
Resource Abstract is supplied to user into virtual resource, the flexibility that can increase the utilization rate of physical resource and utilize.But it is virtual
Change is that virtual management layer is increased between physical hardware and VME operating system, will certainly be to performance shadow
Ring, simultaneously because the features such as the dynamically distributes of virtual resource, multiple users share, compared with physical platform, the void of cloud service platform
Planization technology may result in a certain degree of hydraulic performance decline and unstable, it is therefore desirable to which virtualizing performance is tested.It is empty
The test event of planization performance test is as shown in table 1.
The virtualizing performance of table 1 is tested
Virtual machine is the important representative of cloud service platform virtual resource.Virtual machine in cloud service platform both can be as clothes
Business project is supplied directly to user, it is also possible to as the underlying services device of cloud service for cloud service provide can flexible utilization it is virtual
Resource, therefore it is necessary that the performance of virtual machine test.The test of the reference performance of virtual machine is broadly divided into
Under several aspects：
Processor performance：Processor performance reflect cloud service platform be supplied to user virtual instance the speed of service,
Operational capability, processor performance generally has a very large relationship with the computing capability of virtual machine.By running computation-intensive journey
Sequence, the time for testing completion carrys out the performance of evaluation processor.
Network performance：Different virtual machine is in communication with each other the high-quality network of needs, when multiple virtual machines complete task jointly
It is also required to high-quality virtual machine network communication capacity.From network throughput, network delay, network jitter and data packet loss etc.
Index investigates network performance.
Internal memory performance：Internal memory performance has a significant impact to the speed that virtual machine performs various tasks.By test memory band
Width investigates internal memory performance.
Disk performance：Disk performance all has a great impact to data storage, file read-write.By testing to different size
The order read-write of file, the speed of random read-write investigate disk performance.
Specific embodiment four：Present embodiment is described further to implementation method two, the virtualizing performance test
Test event include virtual machine performance test, cluster virtual machine performance test and virtualizing performance loss test；
The test content of cluster virtual machine performance test includes PC cluster performance test and cluster communication performance test；
The test index of PC cluster performance test is floating number calculating speed；The test index of cluster communication performance test
It is cluster arithmetic speed.
In present embodiment, in many large-scale tasks for the treatment of, it is necessary to many virtual machines carry out common operational, therefore
Need to test the overall performance of cluster virtual machine, it is the calculating of the main integration test cluster virtual machine of this partial test, logical
The performances, the arithmetic speed of test cluster such as letter.
Specific embodiment five：Present embodiment is described further to implementation method two, the survey of virtualizing performance test
Examination project includes virtual machine performance test, cluster virtual machine performance test and virtualizing performance loss test；
The test content of virtualizing performance loss test includes the test with the performance gap with configuration physical machine；
Test index with the test of the performance gap with configuration physical machine includes calculating task process time, network transmission
Speed, internal memory and disk read-write speed, Unixbench total scores.
In present embodiment, due to having virtual machine pipe between physical resource layer and virtual resource layer in cloud service platform
The presence of layer is managed, therefore compared with the physical machine of comparable configuration, the performance of virtual machine has certain loss.This partial test is main
Virtualization is investigated by comparing the virtual machine for similarly configuring in the gap of the aspects such as processor performance, network performance and physical machine
The performance loss for bringing.
Specific embodiment six：Present embodiment is described further to implementation method one, resource management capacity test
Detailed process is：
Step one, cloud service platform send request according to test event to management node；
Step 2, management node are tested according to the corresponding test module of request call, while empty to cloud service platform
Plan machine carries out corresponding configuration；
Step 3, acquisition test index, return to management node.
Specific embodiment seven：Present embodiment is described further to implementation method six, and the resource management capacity is surveyed
The test event of examination includes that the test of cloud service platform autgmentability, the test of virtual machine performance isolation, virtual machine performance stability are surveyed
Examination and virtual machine (vm) migration performance test；
The test content of cloud service platform autgmentability test include virtual resource addition time test, automatic extension test and
Virtual machine extension is tested；
The test index of virtual resource addition time test is the time of heat addition virtual hardware；The survey of automatic extension test
Examination index is from the time for monitoring overload to extension completion；The test index of virtual machine extension test is Unixbench
Total score.
The platform management level of resource management capacity test correspondence cloud service platform, the platform management layer of cloud service platform is negative
Blame and the virtual resource of cloud service platform is scheduled, is managed, realize the dynamic management to various cloud service platform resources, really
Protect the effective and reasonable distribution of resource and ensure that the service level of upper layer application, the performance of whole cloud service platform can be produced very
Big influence, therefore this part is tested.The main managerial ability investigated inside cloud service platform to every resource, test
Item is as shown in table 2：
The resource management capacity of table 2 is tested
Cloud service platform autgmentability is tested：Scalability is a key character of cloud service platform, represents cloud service
The characteristics of platform on-demand service, autgmentability reflects the extended capability of cloud service platform system scale and service, is expanded by testing
Malleability, can predict the performance condition of the system under bigger scale.Because the scheduling mechanism of different cloud service platforms is each not
It is identical, therefore the degree of influence autgmentability is also each different.The autgmentability of cloud service platform can be embodied in many aspects, in money
In the part of source control aptitude tests, following several aspects are mainly tested：
Virtual resource adds the time：Test increases or decreases the time that resource is consumed, and such as adds a virtual machine or gives
Time used by the virtual resources such as existing virtual machine addition CPU, internal memory.
Automatic extension：When the overload that virtual machine is run, cloud service platform can automatically for the virtual machine of user increases
Plus resource, this is a kind of embodiment of cloud service platform resilient expansion ability.Automatic extension has two ways, and one kind is to extend up
(scale up), referring to the resource by increasing single virtual machine (for example increases virutal machine memory capacity, increases core cpu
Number etc.) it is extended；Another kind is, to external expansion (scale out), to refer to the number by increasing virtual machine, is made each
The average load reduction of platform virtual machine, the purpose of resource expansion is reached with this.Test to automatic expansion time, it is main investigate from
The time that overload (certain index being previously set reaches threshold value) terminates to extension.
Virtual machine extension：Two aspects are mainly investigated in virtual machine extension test, and one is that test single virtual machine is
It is no corresponding performance boost to be obtained by resource；Another is the virtual machine quantity that test increases cluster, the performance of cluster
Whether corresponding lifting is had.
Specific embodiment eight：Present embodiment is described further to implementation method six, and the test event includes cloud
The test of service platform autgmentability, the test of virtual machine performance isolation, virtual machine performance stability test and virtual machine (vm) migration performance
Test；
The test content of virtual machine performance isolation test includes the test of computing power isolation, network performance isolation
Test and the test of storage performance isolation, the test index of this three test contents are identical, are that multiple virtual machines interfere with each other feelings
Calculating task process time, network transfer speeds and disk read-write speed under condition.
In present embodiment, virtual machine performance isolation：Intel Virtualization Technology realizes multiple virtual servers and is incorporated into one
Individual physical host, the different virtual machine that there may be multiple users on Same Physical machine improves the utilization rate of resource.But
The various resources of multiple virtual machines meeting contention physical hosts, can have an impact to mutual performance.Virtual machine performance isolation is exactly
For evaluating this index for influenceing each other.Test to virtual machine performance isolation mainly includes the following aspects：
Calculate performance isolation：The resources such as test virtual machine sharing CPU are to calculating the influence that performance is caused.
Network performance isolation：Test virtual machine shares the influence caused to network service performance during network.
Storage performance isolation：Influence during test virtual machine shared storage device to data storage performance.
Specific embodiment nine：Present embodiment is described further to implementation method six, and the test event includes cloud
The test of service platform autgmentability, the test of virtual machine performance isolation, virtual machine performance stability test and virtual machine (vm) migration performance
Test；
The test content of virtual machine performance stability test includes computing power stability test, network performance stability
Test and storage performance stability test；
The test index of computing power stability test is calculating task process time；Network performance stability test is
Network transfer speeds；Storage performance stability test is disk read-write speed.
In present embodiment, virtual machine performance stability：It is different from traditional physical server, due on cloud service platform
Resource by multiple users to share, and virtual resource is that, in dynamic change, the performance of virtual server might have shakiness
Fixed place, being desirable to the performance of virtual machine certainly from user perspective can reach stabilization.Virtual machine performance stability is referred mainly to
Be the performance inconsistency situation that virtual machine under pressure condition higher is kept in the long period.This partial test mainly investigate with
Lower content：
Calculate stability：Test virtual machine calculates performance when computation-intensive program is run within one section of continuous time
Stability.
Network performance stability：The stability of the internal network communication performance between one section of continuous time virtual machine of test.
Storage performance stability：Test virtual machine is stored within one section of continuous time during service data operations-intensive program
The stability of performance.
Specific embodiment ten：Present embodiment is described further to implementation method six, and the test event includes cloud
The test of service platform autgmentability, the test of virtual machine performance isolation, virtual machine performance stability test and virtual machine (vm) migration performance
Test；
The test content of virtual machine (vm) migration performance test includes that active migration performance test and automatic load balancing are tested；
The test index of active migration performance test is the virtual machine (vm) migration time；The test index of automatic load balancing test
It is from monitoring load imbalance again to the time of load balancing.
In present embodiment, virtual machine (vm) migration performance：Virtual machine migration technology can be realized across the virtual of physical server
Change service, virtual machine can switch between different physical servers or physical store, and cloud can be realized based on migrating technology
The high availability of service platform.This partial test mainly investigates the time performance of the dynamic migration operation of virtual machine, so-called dynamic
Migration refers to ensure that the business of virtual machine operation will not on the virtual machine (vm) migration of running status to different physical platforms
Interrupt.The test of this part is divided into active migration test and Autonomic Migration Framework test.
Performance when user independently carries out migration operation is investigated in active migration test, including it is following some：
Single virtual machine (vm) migration：Investigate single virtual machine under different load conditions (load of non-loaded, computation-intensive,
The intensive load of data storage), in the difference of the different storages, the shared storage of different main frames and different main frames of same host
The performance migrated between storage.Time needed for main test migration operation.
Multi-dummy machine is migrated：Consider the migration simultaneously of multiple virtual machines, the target of migration is Same Physical machine or storage.Survey
The migration of (non-loaded, computation-intensive load, the intensive load of data storage) under answering contrast test different loads state during examination
Performance.Time needed for main test migration operation.
Autonomic Migration Framework investigation cloud service platform carries out performance when virtual machine (vm) migration is operated automatically, and main test is following interior
Hold：
Automatic load balancing：So-called automatic load balancing, if run on a certain physical host in referring to cluster
Virtual machine is excessive or virtual machine overload of operation, cloud service platform can it is automatic from the physical host of load weight empty
Plan machine is moved on the physical host of light load in same cluster.User can set a threshold value, when reaching this threshold value
When allow cloud service platform to automatically begin to load balancing operation.This partial test mainly investigate cluster from monitor laod unbalance to
Complete automatic load balancing and operate the spent time.
Specific embodiment 11：Present embodiment is described further to implementation method one, service provision capacity test
Including service ability testing, service extension test and service stability test；
The test index of service ability testing is maximum concurrent user number, response time, number of transactions per second and service failure
Rate；The test index of service extension test is number of transactions per second；The test index of service stability test is the response time.
In present embodiment, three kinds of service modes of cloud service：IaaS, PaaS and SaaS, are inherently certain
Resource services be supplied to user the most.From from the perspective of user, all functions that cloud service platform is provided can be seen as
It is a kind of service.Service provision capacity test is exactly the angle from user, investigates the overall performance of cloud service platform.This part is surveyed
Examination project is as shown in table 3：
The service provision capacity of table 3 is tested
Service ability：The ability that cloud service platform disclosure satisfy that user's request is investigated, cloud service is investigated from many aspects and is put down
The service ability of platform, including following index：
Maximum concurrent user number：The characteristics of cloud service platform is exactly multiple users to share resources, what this partial test was investigated
It is exactly the maximum access number of users that can be carried of cloud service platform.
Response time：Refer to time of the user from a certain service of application to required for obtaining this service.Cloud takes
The response time of platform service of being engaged in can be affected by many factors, the net of system complexity and data center inside cloud service platform
Network communication is all the key factor for influenceing the response time.Response time is divided into maximum response time, Best-case Response Time, average sound
With 90% transaction response time between seasonable.
Number of transactions per second：The cloud service platform number of transactions that can be processed per second is more, illustrates that the service of cloud service platform is carried
Energy supply power is stronger.Number of transactions, the number of transactions for passing through and the number of transactions for failing that number of transactions per second includes submission per second.
Service failure rate：Failure number of transactions accounts for the ratio of total number of transactions.When concurrent user number increases, may be due to network
The reasons such as congestion cause the component requests of user to fail, and service failure rate is exactly to investigate the user's request under different loading conditions
Failure conditions.
Service extension：The autgmentability of cloud service platform is investigated from the angle of the service that provides.Cloud service platform can allow for
Multiple concurrent users simultaneously access, but with number of users increase cloud service platform service performance may under
Drop.This partial test mainly investigates influence of the increase of concurrent user's quantity to cloud service platform performance.
Service stability：Because cloud service platform resource virtualizing, service are by features, the stability of service such as network offers
It is the particularly concerned performance indications of cloud service platform user.It is continuous at one section that this partial test mainly investigates cloud service platform
The stability of service is provided in the time.
In the present invention, the realization of testing tool and testing tool is as follows：
1st, for the test of virtualizing performance, mainly using various testing tools, by a series of benchmark test, from survey
The result of examination measures the performance of each side of virtualization.Tester's initial landing cloud service platform, by control node to
Calculate node asks virtual machine.Calculate node can start virtual machine according to the parameter configuration of the requirement of tester.Tester
The configuring condition of requirement and virtual machine according to test sets every test parameter, afterwards the order deployment testing tool and fortune
The order of row testing tool is packaged into an automatic test script, and test script is uploaded on virtual machine.Transport on a virtual machine
Row test script is tested.In test process, the performance monitoring tool that tester can be by deployment on a virtual machine
Various performance parameters to virtual machine carry out monitor in real time.
Test as shown in table 4 is mainly tested in virtualizing performance test for VMware platforms and CloudStack platforms
Project.The different test event of correspondence and different testing tools, have write different test scripts.In order to ensure test just
True property, another test will not be simultaneously run running when a certain item is tested.
The virtualizing performance test event of table 4
The step of test, is all completed by automatic test script, and the flow in script is as follows：
Testing tool and necessary composing software are obtained from the http-server built in advance；
The required translation and compiling environment of configuration, installs the composing softwares such as gcc, g++, gfortran；
Decompression Wireless Test Toolset simultaneously compiles installation；
According to the parameter testing results being previously set, it is multiple that partial test can set circular flow；
Generation test result file.
During test, the various performance parameters of performance monitoring tools monitor in real time virtual machine.It is main to investigate
The utilization rate of CPU usage and internal memory, can be used to judge the loading condition of virtual machine.Installed and used by virtual server
The performance monitoring instrument Nmon that IBM Corporation releases is monitored to system, while cloud service platform also carries performance monitoring function
To be monitored to cloud service platform physical host performance.
The testing tool used in testing in this section is as shown in table 5：
The virtualizing performance test suite of table 5
| Benchmark | Version | Test content |
| Sysbench | 0.4.1 | Computation-intensive task process time |
| Netperf | 2.6.0 | Handling capacity, data transmission bauds etc. |
| Iperf | 3.0.1 | Network delay, network packet loss rate etc. |
| Stream | 5.10 | Memory bandwidth |
| Iozone | 3.42 | Disk read-write speed |
| HPL/Linpack | 2.1.0 | PC cluster performance/processor performance |
| NPB | 3.3 | PC cluster, communication performance |
| Unixbench | 5.1.2 | Virtualization loss, virtual machine extension |
| Nmon | 14g | Systematic function is monitored |
(1) time required for all prime numbers within the Sysbench value N that certain pre-sets by measuring and calculation come
Investigate and calculate performance.
(2) Netperf is a kind of network service performance measurement tool, there is two kinds of applied in network performance test patterns, and one kind is to criticize
Amount data-transmission mode, test client one-side can send data to service end under this pattern；Another kind is to ask/answer
Answer pattern, client and service end can back and forth transmission information weighing the request/response performance of given packet size.
(3) Iperf is an applied in network performance test instrument, can be postponed with test network, shake and packet loss, the network bandwidth etc.
Situation.
(4) Stream is operated come the performance of test memory for tetra- kinds by copy, scale, add and triad.Using Stream
The memory bandwidth of sustainable operation can be tested out.
(5) Iozone is a disk performance testing tool, can use the read-write of disk in the different operating system of test
Energy.Iozone has multiple testing pattern, and/stressed, sequential write/the test pattern such as rewriting, random read/write is read including order.
(6) Linpack is international high-performance calculation performance test program, and unitary N is solved using Gaussian elimination method
Secondary dense linear algebraic equation systems evaluate the floating number computing capability of computer, HPL be Linpack test programs it is newest simultaneously
Row version.
(7) NPB is international parallel computation benchmark as HPL.NPB is provided by NASA, is one
The performance evaluating program in scientific algorithm field, it is made up of 8 conventional scientific programs, and each test has 6 kinds of scales, no
It is different with the test data set size used by scale.
(8) Unixbench is a comprehensive testing tool for test class Unix operating system performances, there is multiple tests
Project, including Whetstone and Execl Throughput.Result and a benchmark system that Unixbench tests each single item
The test result of system is compared, and has obtained an index value (index), and the index values of all items collect and obtain one
Total test result.
2nd, for the test of cloud service platform resource management capacity, mainly by using every management work(of cloud service platform
Can, different test scenes are set cloud service platform management, scheduling performance are tested.According to the design of different test targets not
Same test scene, then sends request to management node.Management node calls itself corresponding functional module to enter according to request
Row operation, and cloud service platform virtual machine is configured accordingly.The result of test can return to management node, tester
Test result can be checked by logging in management node.In test process, can be by performance monitoring tool monitor in real time cloud
The various performance parameters of service platform.
The test item shown in the main test table 6 of resource management capacity test on VMware platforms and CloudStack platforms
Mesh.In each test event, having write different automatic test scripts makes various testing tools by the prior tester formulated
Draw and perform.In order to ensure the correctness of test, another test will not be simultaneously run when a certain item is tested running.
Some cloud service platforms functions of itself is used to the test of resource management capacity, has been put down by calling cloud service
The certain module of the platform management layer of platform is tested the resource management capacity of cloud service platform.
The resource management capacity of table 6 is tested
For the test of resource management capacity, the main management function using cloud service platform itself, while using some
The performance that load testing instrument comes under test different types load.The testing tool such as table 7 used in testing in this section
It is shown.
When cpu test is carried out, increasing calculation scale can produce higher load to Sysbench to CPU.Netperf can be produced
The substantial amounts of network transmission of life produces load to network.Memtester is a memory pressure testing tool, by carry out XOR,
With or, arithmetic etc. internally deposit into row pressure test, memory usage very high can be reached.Iozone can test disk
Can, disk utilization rate very high is had while test, can be as the loading tool of disk.Nmon is the property that IBM is released
Energy monitoring tools, itself take resource seldom, can check the service condition of CPU, internal memory, the disk of system etc..
The resource management capacity test suite of table 7
| Benchmark | Version | Test content |
| Sysbench | 0.4.1 | Produce load |
| Linpack | 2.1.0 | Test floating number calculating speed |
| Netperf | 2.6.0 | Produce load, testing characteristics of network |
| Memtester | 4.2.2 | Produce load |
| Iozone | 5.10 | Load is produced, disk speed is tested |
| Nmon | 14g | Systematic function is monitored |
3rd, in order to the service provision capacity to different types of cloud service platform is tested, it is necessary to it is a kind of it is widely applicable,
The testing tool that hardware requirement is relatively low, can represent typical cloud service platform service and can simulate live network load.
LoadRunner can be implemented concurrently to load on a large scale, while may be used also with the behavior of analog subscriber to goal systems
Monitor in real time is carried out with to system, is that wide variety of performance testing tool is suffered from worldwide.Therefore the present invention is sharp
A set of cloud service platform load testing instrument has been designed and Implemented with LoadRunner.The testing tool is made using LoadRunner
Be load generating means, the distributed search system based on SOA of independent development as goal systems, using LoadRunner
Analysis and cloud service platform performance monitoring function, can effectively to the service of cloud service platform used as transacter
Carry out load testing.
Cloud service platform load testing instrument based on LoadRunner is that a workload drives, cloud service-oriented
The performance testing tool of platform service performance.The component of the testing tool can be divided into load generation module, goal systems sum
According to collection module, overall Organization Chart is as shown in Figure 3.
Workload generation module realized by LoadRunner test softwares, can produce different scales, dynamic change
Workload.LoadRunner captures customer service flow and creates automation using Virtual User generator (VuGen)
Can test script.LoadRunner consoles (Controller) simulation true environment creates test scene and sets operation load
Plan, controls load generator (Load Generator) to produce the request of virtual client to create load afterwards.Virtual client
The operation of execution is that the script created according to VuGen is performed.
Client occurs from load for data collection module and test run process is collected at cloud service platform virtual machine service end
In application performance situation and resources of virtual machine utilization power data.In client, data collection module by
The Analysis test result analysis device that LoadRunner is carried realizes that the implementing result that it can preserve test scene is simultaneously automatic
Generate related test result chart.In cloud server terminal, the powerful performance monitoring instrument carried using cloud service platform,
Can be to virtual server and the performance indications of host's physical server of virtual machine, such as utilization rate of CPU and internal memory etc. is entered
Row monitoring, the load state at display current service end.
Goal systems is realized using the distributed search system based on SOA framework.Goal systems is built flat in cloud service
On platform, the service that LoadRunner investigates cloud computing platform by simulating a large amount of Virtual User access target systems provides energy
Power.
The workflow of whole testing tool is as follows：Goal systems is built on the virtual server of cloud service platform,
And import test data to back end.Real user behavior is simulated using VuGen create test pin in load generation module
This, Controller formulates load testing plan and controls Load Generator to generate Virtual User and goal systems is surveyed
Examination.In data collection module, cloud service platform performance monitoring tool carries out monitor in real time to the performance indications of server,
Analysis collects test result, and generates test resolution preservation.
Cloud service platform load testing instrument based on LoadRunner has multifrequency nature：Possess high efficiency, Neng Gougao
The utilization client resource of effect, load is directly acted on into goal systems；Possess monitoring property, can correctly reflect that target cloud takes
The soft and hardware performance of business platform；It is easy to transplanting, goal systems is deployed in cloud service platform virtual environment, in identical
Can be transplanted between hypervisor；High concurrent, LoadRunner can simulate the concurrent operations of users up to ten thousand, produce big
The load of amount.Due to possessing these characteristics, the cloud service platform load testing instrument based on LoadRunner is very suitable for cloud
The service provision capacity of service platform is tested.
Claims (6)
1. cloud service platform performance test methods, the cloud service platform includes service interface layer, platform management layer, virtual resource
Layer and physical resource layer, physical resource layer are the physical hardware of bottom, and virtual resource layer is for virtual by the physical resource of bottom
Change, for scheduling virtual machine, monitoring performance and management resource, service interface layer is based on by cloud service platform for platform management layer
Calculation ability is packaged into service interface and is supplied to external user；
The cloud service platform performance test methods, it is characterised in that service of the performance test methods to cloud service platform connects
Mouth layer, platform management layer and virtual resource layer are tested；The method of testing includes virtualizing performance test, resource management capacity
Test and service provide aptitude tests；
Virtualizing performance tests the performance of the virtual resource for testing cloud service platform bottom；
Resource management capacity tests the managerial ability for testing cloud service platform to virtual resource；
Service provision capacity is tested for testing the ability that cloud service platform provides external user service；
The test event of the virtualizing performance test includes virtual machine performance test, cluster virtual machine performance test and virtualization
Performance loss is tested；
The test event of resource management capacity test is tested including cloud service platform autgmentability, virtual machine performance isolation is tested,
Virtual machine performance stability test and virtual machine (vm) migration performance test；
Service provision capacity test includes service ability testing, service extension test and service stability test.
2. cloud service platform performance test methods according to claim 1, it is characterised in that virtualizing performance test it is specific
Process is：
Step one, cloud service platform is logged in, virtual machine is asked to calculate node by control node；
Step 2, calculate node start virtual machine according to the parameter configuration of request；
Step 3, according to the configuration of test event and virtual machine set test parameter；
Step 4, the order of testing tool and the order of testing results instrument are packaged into an automatic test script, and will surveyed
Training is originally uploaded on virtual machine；
Step 5, on a virtual machine testing results script are tested, and obtain test index.
3. cloud service platform performance test methods according to claim 2, it is characterised in that the virtualizing performance test
Test event includes virtual machine performance test, cluster virtual machine performance test and virtualizing performance loss test；
The test content of virtual machine performance test includes processor performance test, applied in network performance test, internal memory performance test and magnetic
Disk performance test；The test index of processor performance test is computation-intensive task process time；The survey of applied in network performance test
Examination index is network delay and packet loss；The test index of internal memory performance test is memory bandwidth；The test of disk performance test
Index is disk read-write speed；
The test content of cluster virtual machine performance test includes PC cluster performance test and cluster communication performance test；Cluster meter
The test index for calculating performance test is floating number calculating speed；The test index of cluster communication performance test is collection group operatione speed
Degree.
4. cloud service platform performance test methods according to claim 1, it is characterised in that the tool of resource management capacity test
Body process is：
Step one, cloud service platform send request according to test event to management node；
Step 2, management node are tested according to the corresponding test module of request call, while to cloud service platform virtual machine
Carry out corresponding configuration；
Step 3, acquisition test index, return to management node.
5. cloud service platform performance test methods according to claim 4, it is characterised in that the survey of resource management capacity test
Examination project includes the test of cloud service platform autgmentability, the test of virtual machine performance isolation, virtual machine performance stability test and void
Plan machine migration performance is tested；
The test content of virtual machine performance isolation test includes the test of computing power isolation, the test of network performance isolation
Tested with storage performance isolation, the test indexs of this three test contents are identical, are in the case of multiple virtual machines are interfered with each other
Calculating task process time, network transfer speeds and disk read-write speed；
The test content of virtual machine performance stability test includes computing power stability test, network performance stability test
With storage performance stability test；The test index of computing power stability test is calculating task process time；Internetworking
Stabilizability test is network transfer speeds；Storage performance stability test is disk read-write speed；
The test content of virtual machine (vm) migration performance test includes that active migration performance test and automatic load balancing are tested；Actively move
The test index for moving performance test is the virtual machine (vm) migration time；The test index of automatic load balancing test is from monitoring load
The unbalance heavy time to load balancing.
6. cloud service platform performance test methods according to claim 1, it is characterised in that service provision capacity test includes
Service ability testing, service extension test and service stability test；
The test index of service ability testing is maximum concurrent user number, response time, number of transactions per second and service crash rate；Clothes
The test index of business autgmentability test is number of transactions per second；The test index of service stability test is the response time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410614010.4A CN104333488B (en) | 2014-11-04 | 2014-11-04 | Cloud service platform performance test methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410614010.4A CN104333488B (en) | 2014-11-04 | 2014-11-04 | Cloud service platform performance test methods |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104333488A CN104333488A (en) | 2015-02-04 |
| CN104333488B true CN104333488B (en) | 2017-06-20 |
Family
ID=52408146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410614010.4A Active CN104333488B (en) | 2014-11-04 | 2014-11-04 | Cloud service platform performance test methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104333488B (en) |
Families Citing this family (52)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016048394A1 (en) | 2014-09-25 | 2016-03-31 | Hewlett Packard Enterprise Development Lp | Testing a cloud service |
| CN104639402A (en) * | 2015-03-05 | 2015-05-20 | 浪潮电子信息产业股份有限公司 | Method for server cluster system network test |
| CN104735063B (en) * | 2015-03-11 | 2018-01-02 | 广东电子工业研究院有限公司 | A kind of safe evaluating method for cloud infrastructure |
| CN104794058B (en) * | 2015-05-07 | 2017-10-10 | 合肥中科云巢科技有限公司 | A kind of method of testing of cloud desktop virtual environments performance |
| CN105141473B (en) * | 2015-08-10 | 2018-08-24 | 机械工业仪器仪表综合技术经济研究所 | A kind of online analytical instrument communication reliability evaluating method and system |
| CN105187269A (en) * | 2015-08-14 | 2015-12-23 | 上海交通大学 | Method for evaluating performances of cloud platform based on performance model |
| CN105357067A (en) * | 2015-10-14 | 2016-02-24 | 广州杰赛科技股份有限公司 | Test method and system for cloud platform |
| CN105281979A (en) * | 2015-10-29 | 2016-01-27 | 贵州电网有限责任公司电力调度控制中心 | Load test method for virtualization application platform |
| CN105446846B (en) * | 2015-11-30 | 2018-07-10 | 中电科华云信息技术有限公司 | Performance test methods based on cloud desktop |
| CN105630575B (en) * | 2015-12-23 | 2019-02-15 | 兰雨晴 | For the performance estimating method of KVM virtualization server |
| CN111258867B (en) * | 2015-12-30 | 2022-04-22 | 华为技术有限公司 | Public cloud dial testing method and device |
| CN105847088A (en) * | 2016-05-18 | 2016-08-10 | 浪潮电子信息产业股份有限公司 | Virtual machine performance test system based on cloud service |
| CN106445759A (en) * | 2016-10-28 | 2017-02-22 | 浪潮软件集团有限公司 | Universal cloud server testing method |
| CN107070981B (en) * | 2017-01-23 | 2020-02-04 | 努比亚技术有限公司 | Multi-terminal equipment cooperative control system and method |
| CN107257290B (en) * | 2017-05-15 | 2023-01-24 | 中国电力科学研究院 | Test method and system for open SOA service-oriented architecture |
| CN107220126B (en) * | 2017-05-27 | 2020-12-01 | 南方电网调峰调频发电有限公司 | X86 server dynamic hard partition method, device, storage medium and computer equipment |
| CN107168880A (en) * | 2017-05-31 | 2017-09-15 | 中标软件有限公司 | Virtual machine method of testing and instrument |
| CN107547261B (en) * | 2017-07-24 | 2020-10-27 | 华为技术有限公司 | Cloud platform performance test method and device |
| CN109491754A (en) * | 2017-09-12 | 2019-03-19 | 北京京东尚科信息技术有限公司 | The performance test methods and device of virtual server |
| US10838840B2 (en) | 2017-09-15 | 2020-11-17 | Hewlett Packard Enterprise Development Lp | Generating different workload types for cloud service testing |
| CN107704370A (en) * | 2017-09-28 | 2018-02-16 | 四川长虹电器股份有限公司 | Affairs successful rate statistics method based on response time and the double dimensions of running status |
| CN107707663A (en) * | 2017-10-17 | 2018-02-16 | 长沙准光里电子科技有限公司 | A kind of virtual resources integration platform |
| CN107908512A (en) * | 2017-11-14 | 2018-04-13 | 郑州云海信息技术有限公司 | The test method and test device of a kind of disk performance |
| CN107979876B (en) * | 2017-11-16 | 2022-01-07 | 广州市百果园网络科技有限公司 | Data transmission method and terminal |
| CN108011779A (en) * | 2017-11-30 | 2018-05-08 | 四川长虹电器股份有限公司 | The test method of Cloud Server task throughput under limited resources supplIes |
| CN108763042A (en) * | 2018-05-24 | 2018-11-06 | 广东睿江云计算股份有限公司 | A kind of Cloud Server performance data acquisition method and device based on python |
| CN108923997B (en) * | 2018-05-24 | 2021-07-20 | 广东睿江云计算股份有限公司 | Cloud service node automatic testing method and device based on python |
| CN108900324B (en) * | 2018-06-13 | 2021-10-15 | 平安科技（深圳）有限公司 | Method and device for checking communication performance of virtual machine |
| CN109302305A (en) * | 2018-08-08 | 2019-02-01 | 杭州安恒信息技术股份有限公司 | A kind of clustering performance test method based on industrial control system |
| CN108696448B (en) * | 2018-08-16 | 2021-07-23 | 安徽云才信息技术有限公司 | Service current limiting degradation method based on gateway SOA architecture |
| CN109327553A (en) * | 2018-12-06 | 2019-02-12 | 郑州云海信息技术有限公司 | A kind of operation management system and method towards IaaS cloud platform |
| CN109728938A (en) * | 2018-12-11 | 2019-05-07 | 国云科技股份有限公司 | A kind of method of assessment system service level |
| CN109800113A (en) * | 2018-12-29 | 2019-05-24 | 北京奇安信科技有限公司 | One kind being based on virtual platform performance test methods, system, equipment and medium |
| CN110247817B (en) * | 2019-05-13 | 2021-02-12 | 华中科技大学 | Method and system for measuring performance interference between virtual network functions |
| CN110457197B (en) * | 2019-08-19 | 2023-05-09 | 中国工商银行股份有限公司 | Service testing method, device, computer equipment and storage medium |
| CN110716875A (en) * | 2019-09-26 | 2020-01-21 | 北京计算机技术及应用研究所 | Concurrency test method based on feedback mechanism in domestic office environment |
| CN111224848B (en) * | 2020-01-17 | 2022-04-01 | 腾讯科技（深圳）有限公司 | Network quality testing method, device, equipment and storage medium |
| CN111447103B (en) * | 2020-03-09 | 2022-01-28 | 杭州海康威视***技术有限公司 | Virtual device management system, electronic device, virtual device management method, and medium |
| CN111382080A (en) * | 2020-03-16 | 2020-07-07 | 深圳市吉祥腾达科技有限公司 | Stability test method for equipment cloud management platform system |
| CN111612373B (en) * | 2020-05-29 | 2023-06-30 | 杭州电子科技大学 | Public cloud system performance consistency adjustment method |
| CN112162891B (en) * | 2020-10-14 | 2024-05-03 | 腾讯科技（深圳）有限公司 | Performance test method in server cluster and related equipment |
| CN112346977A (en) * | 2020-11-10 | 2021-02-09 | 中国信息通信研究院 | Quantum cloud computing platform software function evaluation method and device |
| CN112579399B (en) * | 2020-12-28 | 2024-04-09 | 上海蓝云网络科技有限公司 | Cloud service testing method and device, electronic equipment and computer storage medium |
| CN113160976A (en) * | 2021-04-29 | 2021-07-23 | 上海商汤智能科技有限公司 | Medical data processing method and device based on SaaS service and electronic equipment |
| CN113507399B (en) * | 2021-07-09 | 2022-07-26 | 西安电子科技大学 | Network performance evaluation device and method for different levels of cloud platform |
| CN113434281B (en) * | 2021-07-19 | 2024-05-28 | 上海幻电信息科技有限公司 | Equipment scheduling method and cloud platform |
| CN113656161A (en) * | 2021-08-17 | 2021-11-16 | 中正信评(深圳)技术服务有限公司 | Method, device and system for scheduling pressure generation service in performance test |
| CN113609027A (en) * | 2021-08-31 | 2021-11-05 | 北京百度网讯科技有限公司 | ARM cloud server testing method and device, electronic equipment and storage medium |
| CN114257527B (en) * | 2021-11-01 | 2024-02-02 | 北京思特奇信息技术股份有限公司 | Network bearing capacity estimation method |
| CN114070754B (en) * | 2021-11-15 | 2023-06-23 | 迈普通信技术股份有限公司 | Method and system for testing migration performance of Overlay network virtual machine |
| CN115277471A (en) * | 2022-08-01 | 2022-11-01 | 珠海金山数字网络科技有限公司 | Index monitoring method, system and device and electronic equipment |
| CN117891619B (en) * | 2024-03-18 | 2024-06-11 | 山东吉谷信息科技有限公司 | Host resource synchronization method and system based on virtualization platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101969475A (en) * | 2010-11-15 | 2011-02-09 | 张军 | Business data controllable distribution and fusion application system based on cloud computing |
| CN101986280A (en) * | 2010-11-29 | 2011-03-16 | 浙江大学 | Automatic testing platform for virtual computing system |
| WO2013006708A1 (en) * | 2011-07-07 | 2013-01-10 | Vce Company, Llc | System of automatic monitoring of a physical computing platform and just- in-time allocation of resources |
-
2014
- 2014-11-04 CN CN201410614010.4A patent/CN104333488B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101969475A (en) * | 2010-11-15 | 2011-02-09 | 张军 | Business data controllable distribution and fusion application system based on cloud computing |
| CN101986280A (en) * | 2010-11-29 | 2011-03-16 | 浙江大学 | Automatic testing platform for virtual computing system |
| WO2013006708A1 (en) * | 2011-07-07 | 2013-01-10 | Vce Company, Llc | System of automatic monitoring of a physical computing platform and just- in-time allocation of resources |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104333488A (en) | 2015-02-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104333488B (en) | Cloud service platform performance test methods | |
| Zhang et al. | {History-Based} harvesting of spare cycles and storage in {Large-Scale} datacenters | |
| Xavier et al. | A performance comparison of container-based virtualization systems for mapreduce clusters | |
| Grozev et al. | Performance modelling and simulation of three-tier applications in cloud and multi-cloud environments | |
| Novaković et al. | {DeepDive}: Transparently identifying and managing performance interference in virtualized environments | |
| Ou et al. | Is the same instance type created equal? exploiting heterogeneity of public clouds | |
| Anand et al. | Virtual machine placement optimization supporting performance SLAs | |
| CN104380260B (en) | Utilize the reservoir simulation of scalable grid computing | |
| US10379883B2 (en) | Simulation of high performance computing (HPC) application environment using virtual nodes | |
| Huang et al. | Virt-LM: a benchmark for live migration of virtual machine | |
| Hirofuchi et al. | Simgrid vm: Virtual machine support for a simulation framework of distributed systems | |
| Lloyd et al. | Migration of multi-tier applications to infrastructure-as-a-service clouds: An investigation using kernel-based virtual machines | |
| Mondesire et al. | Combining virtualization and containerization to support interactive games and simulations on the cloud | |
| Ates et al. | HPAS: An HPC performance anomaly suite for reproducing performance variations | |
| El-Refaey et al. | CloudGauge: a dynamic cloud and virtualization benchmarking suite | |
| Patros et al. | Investigating resource interference and scaling on multitenant PaaS clouds | |
| Galante et al. | A programming-level approach for elasticizing parallel scientific applications | |
| Wang et al. | Performance comparison between hypervisor-and container-based virtualizations for cloud users | |
| Yan et al. | Optimizing power and performance trade-offs of MapReduce job processing with heterogeneous multi-core processors | |
| Neves et al. | Black-box inter-application traffic monitoring for adaptive container placement | |
| Netto et al. | Evaluating load generation in virtualized environments for software performance testing | |
| Popiolek et al. | Monitoring and analysis of performance impact in virtualized environments | |
| Ye et al. | Virt-B: Towards performance benchmarking of virtual machine systems | |
| El-Refaey et al. | Virtual systems workload characterization: An overview | |
| Yan et al. | Dyscale: a mapreduce job scheduler for heterogeneous multicore processors |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |