CN110427323B

CN110427323B - Application testing method, device, proxy server and system

Info

Publication number: CN110427323B
Application number: CN201910690397.4A
Authority: CN
Inventors: 张慧吉; 叶明登; 施晓东; 何宇
Original assignee: Tianjin Chezhijia Data Information Technology Co ltd
Current assignee: Tianjin Chezhijia Data Information Technology Co ltd
Priority date: 2019-07-29
Filing date: 2019-07-29
Publication date: 2022-12-02
Anticipated expiration: 2039-07-29
Also published as: CN110427323A

Abstract

The invention discloses an application testing method, which is suitable for being executed in a proxy server and comprises the following steps: receiving a page starting request for a certain page in a target application, which is sent by a client, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time; intercepting response contents of each interface request, wherein each response content comprises a plurality of fields, and the fields have a hierarchical relationship; uniformly modifying the same layer of fields in each response content into a certain type of boundary value according to a preset rule, and returning the modified response value to the client for operation; and judging whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if so, automatically associating the fault information with a corresponding response value. The invention also discloses a corresponding application testing device and a corresponding application testing system.

Description

Application testing method, device, proxy server and system

Technical Field

The invention relates to the field of automatic testing, in particular to an application testing method, an application testing device, a proxy server and an application testing system.

Background

With the increasing popularization of intelligent mobile terminals, more and more developers are put into the application development of mobile terminal equipment, and application testing is an indispensable important link in the application development process. Whether the application has errors or not, and whether the running process or the running result meets the expected requirements or not can be known through application testing. The robustness test is an application test and is mainly used for testing whether a system can automatically recover or continue to operate by ignoring a fault when the system fails.

The robustness testing applied is usually done in two ways. One is to write unit tests, such as android, to perform mock on a network request method through a third-party open source framework such as PowerMock. However, the unit test is only suitable for codes with small dependence on interface context, and for methods in components with strong dependence on context, such as Activity, service and the like, the method is mainly a private method and related to the life cycle of the private method, and the unit test cannot be directly performed. Moreover, this approach is relatively costly and requires extensive code logic. The other method is to realize the package grabbing mock data by a package grabbing tool such as Charles, fiddle and the like, fields in the interfaces need to be modified one by one manually, the cost is very high, an application generally requests a plurality of interfaces simultaneously, the manual work of the mock is easy to exceed the time of application request, once one more quotation mark or comma is added, an error is caused, and the value and the scene of the mock need to be recorded manually after a problem is found, the bug recording cost is very high, and the mock is not easy to archive.

Disclosure of Invention

In view of the above, the present invention provides an application testing method, apparatus, proxy server and system, which seek to solve, or at least solve, the above-mentioned problems.

According to an aspect of the present invention, there is provided an application testing method adapted to be executed in a proxy server, the method comprising the steps of: receiving a page starting request for a certain page in a target application, which is sent by a client, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time; intercepting response contents of each interface request, wherein each response content comprises a plurality of fields, and the fields have a hierarchical relationship; uniformly modifying the same layer of fields in each response content into a certain type of boundary value according to a preset rule, and returning the modified response value to the client for operation; and judging whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if so, automatically associating the fault information with a corresponding response value.

Optionally, in the application test method according to the invention, the type of the boundary value comprises at least one of: random string, super-long string, special string, integer, single precision floating point, double precision floating point, time, array, boolean, url, mailbox, IP, picture, expression, null array, null object, null, undefined.

Optionally, in the application testing method according to the present invention, the predetermined rule includes: modifying the field of each response content layer by layer, modifying the same layer of fields of all response contents each time, and modifying the layer of fields into boundary values of the same type each time; and modifying the next-layer fields of all the response contents after traversing all the types of boundary values of the fields of one layer.

Optionally, in the application testing method according to the present invention, after the modified response value is returned to the client, the method further includes the steps of: and sending the unique task identifier, the page identifier, the modified hierarchy and boundary value type, the equipment identifier and the operation starting time which are started at this time to a message queue, and considering that a page test task is completed at this time.

Optionally, in the application testing method according to the present invention, the target application includes a plurality of page identifiers, and the method further includes the steps of: and after the page test task is completed once, cleaning the test environment of the target application, and performing page test tasks of other page identifications or other levels or other boundary value types until the page test tasks of all boundary value types of all levels under all page identifications are completed.

Optionally, in the application test method according to the present invention, the page activation request is sent in a schema manner, and a field of each response content is divided into at most ten levels.

Optionally, in the application testing method according to the present invention, the proxy server is communicatively connected to a plurality of clients, each client including the target application, and the method further includes the steps of: and distributing the plurality of page identifications to a plurality of clients, and storing an association relation table of the device identifications of the clients and the distributed page identifications so that each client can execute a page test task corresponding to each page identification.

Optionally, in the application testing method according to the present invention, further comprising the steps of: and distributing the multiple types of the boundary values to multiple clients, and storing an association relation table of the equipment identification of each client and the distributed boundary value type so that each client can execute boundary value test of the corresponding type on each page.

Optionally, in the application testing method according to the present invention, the interface request includes a service class interface request and a non-service class interface request, and the proxy server only modifies response content of the service class interface request.

Optionally, in the application testing method according to the present invention, a domain name white list of the service class interface is further stored in the proxy server, and the method further includes the steps of: and acquiring a domain name value in a request header of each interface request, and identifying the service type interface request according to a domain name white list.

Optionally, in the method according to the present invention, the traffic class interface is logically associated with an application interface, and the non-traffic class interface includes a buried point interface and an advertisement exposure request interface.

Optionally, in the application testing method according to the present invention, further comprising the steps of: and constructing a special boundary test packet, and sending the special boundary test packet to a client for installation, wherein the special boundary test packet supports a mock function, an interface log uploading function and an interface response uploading function.

Optionally, in the application testing method according to the present invention, the failure information reported by the client during the operation process includes a unique task identifier when a failure occurs, a service interface request, and a corresponding response value.

Optionally, in the application testing method according to the present invention, the fault information is reported in JSON format, and further includes at least one of a device identifier, a network status, application version information, a package name, and exception stack information.

Optionally, in the application testing method according to the present invention, further comprising the steps of: consuming the message queue, taking out the unique task identifiers one by one, and inquiring whether corresponding fault information exists according to the unique task identifiers; if so, judging that the boundary value of the operation is abnormal, automatically associating the fault information with the corresponding message queue information, and sending the fault information to a tester.

According to another aspect of the present invention, there is provided an application testing method including: a client sends a page starting request for a certain page in a target application to a service server through a proxy server, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time; the method comprises the steps that a proxy server intercepts response contents of interface requests returned by a service server, wherein each response content comprises a plurality of fields, and the fields have a hierarchical relationship; the proxy server uniformly modifies the same layer of fields in each response content into a certain type of boundary value according to a preset rule and returns the modified response value to the client; the client operates the response value and automatically reports the current fault information to the service server when a fault occurs, wherein the fault information comprises the current unique task identifier; and the proxy server judges whether the unique task identifier has corresponding fault information from the reported records, and if so, automatically associates the fault information with the corresponding response value.

According to another aspect of the present invention, there is provided an application testing apparatus adapted to reside in a proxy server, the apparatus comprising: the request receiving module is suitable for receiving a page starting request for a certain page in the target application, which is sent by a client, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time; the response intercepting module is suitable for intercepting response contents requested by each interface, and each response content comprises a plurality of fields which have a hierarchical relationship; the response modification module is suitable for uniformly modifying the same layer of fields in each response content into a certain type of boundary value according to a preset rule and returning the modified response value to the client for operation; and the fault analysis module is suitable for judging whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if so, automatically associating the fault information with the corresponding response value.

According to still another aspect of the present invention, there is provided a proxy server including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs when executed by the processors implement the steps of the application testing method as described above.

According to a further aspect of the present invention there is provided a readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a proxy server, carry out the steps of the application testing method as described above.

According to the technical scheme of the invention, the network request of the application terminal is intercepted, and the response of the interface is modified into an irregular data return value according to a preset rule. And only modifying one boundary value type under one response level when mock modification is carried out every time, and traversing to the abnormal logic of the application end as much as possible through modifying interface response in a layering way. On the other hand, the invention automatically collects the fault (blast) information of the application end and automatically correlates with the modified mock data, thereby facilitating testers to find defects and loopholes in the program in time, reducing abnormity and reducing the labor cost of the robustness test. Moreover, the invention also defines a plurality of (such as 18) types for the possible numerical boundary values of the application end, and supports only single equipment mock and only business class interface mock without influencing the normal use of other users.

Furthermore, the invention sets the schema for each page of the application terminal, and can directly open any page, thereby minimizing the execution cost of the arriving page and improving the stability of the automatic test. The running record, the crash information, the interface request and the response data can be associated according to the unique task identifier, so that the problem reason can be conveniently positioned. The scheme can be operated in a fully automatic mode without manual operation, and the designed mock rule is applicable to all interfaces and is universal without adaptation according to specific interfaces, so that the maintenance cost is extremely low.

The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.

Drawings

To the accomplishment of the foregoing and related ends, certain illustrative aspects are described herein in connection with the following description and the annexed drawings, which are indicative of various ways in which the principles disclosed herein may be practiced, and all aspects and equivalents thereof are intended to be within the scope of the claimed subject matter. The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description read in conjunction with the accompanying drawings. Throughout this disclosure, like reference numerals generally refer to like parts or elements.

FIG. 1 shows a block diagram of an application testing system 100 according to one embodiment of the invention;

FIG. 2 shows a block diagram of a computing device 200, according to one embodiment of the invention;

FIG. 3 illustrates a flow diagram of an application testing method 300 performed in a proxy server according to one embodiment of the present invention;

FIG. 4 is a diagram illustrating a plurality of field hierarchies for interface response content according to one embodiment of the present invention;

FIG. 5 shows a flow diagram of an application testing method 500 according to another embodiment of the invention;

FIG. 6 illustrates a flow diagram of an application testing method 600 performed in an application testing system according to yet another embodiment of the present invention; and

FIG. 7 illustrates a block diagram of an application testing apparatus 700 according to one embodiment of the invention;

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

FIG. 1 shows a schematic diagram of an application testing system 100 according to one embodiment of the invention. As shown in FIG. 1, the application testing system 100 includes one or more clients 110 (e.g., clients 1-3, etc.), a proxy server 120, and a service server 130. The proxy server 120 is communicatively connected to the client 110 and the service server 130, respectively, for forwarding requests and responses between the client 110 and the service server 130. According to one embodiment, the proxy server 120 may be an anyproxy proxy server.

Each client 110 is installed with a target application, and is imported with a trusted https certificate to implement a mock interface function, and a proxy server mechanism is further provided in the client 110. So that all requests and responses sent by the client 110 are forwarded through the proxy server 120. The client 110 also has a boundary test dedicated packet installed therein, which supports a mock function and an interface log (including interface request and response) uploading function. The interface log uploading is controlled by a switch of a network request component of the client 110, the interface log uploading switch is turned on by the boundary test special packet, and each request and response is uploaded to the back-end service server 130.

Generally, a target application comprises a plurality of application pages, in order to ensure the stability and the high efficiency of application testing, the invention sets a schema for each page, any page can be directly started through the schema, and when the schema is started, a unique task identifier is generated for the starting and is transmitted to the target application as a schema parameter. Here, scheme is an intra-page jump protocol, by which it is very convenient to jump to each page in an application, and the format is, for example, autohome:// cardetail/goodsDetailtask _ id = b27214cba92b11e99bd090b11c30468d.

It should be understood that a page may depend on multiple interfaces at the same time, and thus the page initiation request sent by the client 110 will typically include one or more interface requests. These interfaces may include traffic class interfaces and non-traffic interfaces. The business interface, i.e. the field returned by the interface, is related to the application interface logic, and the non-business interfaces, such as a buried point request interface, an advertisement exposure request interface and the like, are mainly used for operation statistics and verification and marketing advertisement, and have no direct relation with the application normal interface display. When the application test is performed, the proxy server 120 adopts a excluding manner for the response of the non-service interface request, which only modifies the response content of the service interface and directly passes the response content of the non-service interface.

That is, when receiving the interface requests, the proxy server 120 identifies the service interface requests therein, modifies the mock response of the service interface requests, and returns the mock response to the client 110 for operation. In general, the proxy server 120 may store a domain name white list of traffic class interfaces, taking into account the HOST field value in the request header attribute of the interface request (https) as the domain name of the current request. Therefore, when the proxy server 120 receives the interface request, it may obtain the domain name value in the request header of the interface request, identify the service class interface request therein according to the domain name white list, and modify the response content of the service class interface request. If the HOST field is a non-service interface domain name, such as a buried point interface domain name or an advertisement interface domain name, then the mock logic is not executed and the original responses of the buried point interface and the advertisement interface are returned to the client 110. After the responses of the non-service interface requests are eliminated, the number of the requests needing mock rule processing is greatly reduced, and the boundary test efficiency is improved.

The service server 130 may process the request and return the request response, and may collect the fault (blast) information that the client 110 automatically reports itself during the operation process, where the fault information generally includes the current operating device status and the processing request and response.

It should be noted that the application testing system 100 of fig. 1 is only exemplary, and in a specific practical situation, there may be different numbers of the proxy servers 120 and the service servers 130 in the system 100, and the present invention does not limit the number of the devices included in the system 100. Moreover, the proxy server 120 and the service server 130 may be a single server, or may be a server cluster formed by a plurality of servers, or may be a cloud computing service center, and a plurality of servers forming the server cluster or the cloud computing service center may reside in a plurality of geographic locations.

According to an embodiment of the present invention, the system 100 may further include one or more clients 140 (e.g., clients i-iii, etc.), and the clients 140 are not provided with a proxy server mechanism, so that the clients 140 may be directly connected to the service server 130 in a communication manner, that is, may directly send access requests to the service server 130 and receive response contents returned therefrom, without forwarding the requests and responses through the proxy server 120. Therefore, the invention only carries out application tests (such as boundary tests, abnormal detection and other robustness tests) on part of equipment provided with the proxy mechanism, and does not influence the normal use of other users.

According to an embodiment of the present invention, the proxy server 120 and the service server 130 in the application testing system 100 can be implemented by the computing device 200 as described below. FIG. 2 illustrates a block diagram of a computing device 200, according to one embodiment of the invention.

In a basic configuration 202, computing device 200 typically includes system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between the processor 204 and the system memory 206.

Depending on the desired configuration, the processor 104 may be any type of processing, including but not limited to: a microprocessor (μ P), a microcontroller (μ C), a Digital Signal Processor (DSP), or any combination thereof. The processor 204 may include one or more levels of cache, such as a level one cache 210 and a level two cache 212, a processor core 214, and registers 216. Example processor core 214 may include an Arithmetic Logic Unit (ALU), a Floating Point Unit (FPU), a digital signal processing core (DSP core), or any combination thereof. The example memory controller 218 may be used with the processor 204, or in some implementations the memory controller 218 may be an internal part of the processor 204.

Depending on the desired configuration, system memory 206 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 106 may include an operating system 220, one or more applications 222, and program data 224. In some embodiments, application 222 may be arranged to operate with program data 124 on an operating system. Program data 124 includes instructions and in computing device 200 according to the present invention, program data 224 includes instructions for performing application testing method 300 and/or 500.

Computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via the bus/interface controller 230. The example output device 242 includes a graphics processing unit 248 and an audio processing unit 250. They may be configured to facilitate communication with various external devices, such as a display or speakers, via one or more a/V ports 252. Example peripheral interfaces 244 can include a serial interface controller 254 and a parallel interface controller 256, which can be configured to facilitate communications with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., printer, scanner, etc.) via one or more I/O ports 258. An example communication device 246 may include a network controller 260, which may be arranged to facilitate communications with one or more other computing devices 262 over a network communication link via one or more communication ports 264.

The network communication link may be one example of a communication medium. Communication media may typically be embodied by computer readable instructions, data structures, program modules, and may include any information delivery media, such as carrier waves or other transport mechanisms, in a modulated data signal. A "modulated data signal" may be a signal that has one or more of its data set or its changes made in such a manner as to encode information in the signal. By way of non-limiting example, communication media may include wired media such as a wired network or private-wired network, and various wireless media such as acoustic, radio Frequency (RF), microwave, infrared (IR), or other wireless media. The term computer readable media as used herein may include both storage media and communication media.

Computing device 200 may be implemented as a server, such as a file server, a database server, an application server, a WEB server, etc., or as part of a small-form factor portable (or mobile) electronic device, such as a cellular telephone, a Personal Digital Assistant (PDA), a wireless WEB-watch device, an application specific device, or a hybrid device that include any of the above functions. Computing device 200 may also be implemented as a personal computer including both desktop and notebook computer configurations. In some embodiments, the computing device 200 is configured to execute the application testing method 300 and/or 500.

FIG. 3 shows a flow diagram of an application testing method 300 according to one embodiment of the invention. The method 300 is performed in a proxy server, such as the proxy server 120, to perform tests on a target application, such as boundary tests and anomaly detection robustness tests.

As shown in fig. 3, the method begins at step S310. In step S310, a page start request sent by a client for a certain page in a target application is received, where the page start request includes a unique task identifier and one or more interface requests of this start. Any page can be started in a schema mode, and a page starting request is sent, so that operation according to a page path of an application is not needed, and each page can be directly started.

Subsequently, in step S320, response contents requested by each interface are intercepted, each response content includes a plurality of fields, and the plurality of fields have a hierarchical relationship.

Here, the proxy server intercepts the response content of the service interface request for modification, and directly passes through the response content of the non-service interface. Fig. 4 is a schematic diagram illustrating interface response contents according to an embodiment of the present invention, where the response contents adopt JSON format, there are multiple levels of data interfaces, level 2 is nested in level 1, level 3 is nested in level 2, and so on, and at most ten levels can be generally divided, and specifically, the interface response contents can be divided into five levels, some six levels, and some ten levels. Exemplary 6 hierarchical relationships are labeled in FIG. 4, such as including message and result fields in level 1, bsdata, bucard, bupkcard, and focusing fields in level 2, and so on. It should be understood that if a field appears in the subsequent content in a parallel relationship with level 1, it is also level 1. Similarly, there may be level 2 parallel fields in the entire response, which also belong to level 2, and so on.

Subsequently, in step S330, the same layer field in each response content is uniformly modified to a certain type of boundary value according to a predetermined rule, and the modified response value is returned to the client for operation.

According to some embodiments of the invention, the boundary values may comprise at least one of the following types: random string, very long string, special string, integer, single precision floating point, double precision floating point, time, array, boolean, url, mailbox, IP, picture, expression, null, undefined. The predetermined rules include: modifying the field of each response content layer by layer, modifying the same layer of field of all response contents each time, and modifying the layer of field into the same type of boundary value each time; and modifying the next layer of fields of all response contents after traversing all types of boundary values of the first layer of fields.

According to other embodiments of the present invention, after the proxy server returns the modified response value to the client, the unique task identifier, the page identifier, the modified hierarchy and boundary value type, the device identifier, and the operation start time that are started or modified at this time are also sent to the message queue, and at this time, it is considered that a page test task is completed. The following are examples of messages sent by a task queue to a message queue:

after the page test task is completed once, the test environment of the target application is cleaned (in order to ensure that the execution is independent and not interfered every time, the application cache is cleaned after the operation is completed every time), and the page test tasks of other page identifications or other levels or other boundary value types are carried out until the page test tasks of all boundary value types of all levels under all the page identifications are completed.

That is, the present invention performs application boundary test page by page, that is, a specified page in an application is made to run under various predetermined boundary value conditions, and an interface log and fault information of an application terminal are automatically captured during the running process. For a certain target page, the page is started for the first time in a schema mode, and then the proxy server intercepts the response content of the service interface in the page starting request to modify the mock. When modifying, firstly, uniformly modifying the fields of the level 1 of all the business interfaces into a type of boundary value, and then returning the type of boundary value to the client for operation, if uniformly modifying the type of boundary value into a random character string.

And then, after the test environment is cleaned, performing second page starting on the target page, and at the moment, intercepting the response of the service interface in the second page starting request again by the proxy server for modification. As shown in Table 1, upon this second modification, the level 1 field may be modified to a second type of boundary value. And so on until all the boundary value types of the level 1 are traversed. For a level, if there are 18 types of boundary values, 18 page launches and 18 responsive content intercepts and modifications will occur. And then, starting to test the boundary value of the level 2, and when all types of boundary values of the level 2 are traversed, then testing the boundary value of the level 3, and so on.

Table 1 application test options example

And when all boundary value types of all levels of one page are mock, performing boundary test on the next target page, and similarly, performing level-by-level test. That is, each time a row configuration in table 1 is selected for modification, until each row in table 1 is traversed. The layer-by-layer type mcok mode can fully mine potential fault information of the application end, and potential problems in each layer of processing logic of the interface of the application end are guaranteed to be exposed. The single-level and single-type modification also facilitates the development personnel to quickly realize fault location and improve the test efficiency.

Taking the url field of level 6 in fig. 4 as an example, when 18 types of boundary value modifications are performed, the normalized http is modified to the irregular values in table 2 to observe whether the application fails.

TABLE 2 boundary values of the type 18

For the boundary test, when the field type agreed by the application is integer, the range of the shaping value is-65536 to-65536, and then the abnormal values of 65537, null, 5.5 and the like are respectively assigned when the field type is mock, and then whether the application end fails or not is observed. And when the field type of the application convention is a character string type, assigning the character string to be the values of a special character, an ultra-long text, an emoticon, a null character and the like during mock, and then observing whether the application end fails or not.

In view of the need to test multiple pages and multiple types of boundary values, in one implementation, multiple page identifiers may be allocated to multiple clients, and an association table between device identifiers of the clients and the allocated page identifiers is stored, so that each client performs a page test task corresponding to each page identifier. In this way, each client is responsible for testing the boundaries of several pages, and after the testing of one page is completed, the testing of the next page identifier is automatically started. When the allocation is performed, the allocation can be performed evenly according to the number of the device identifiers and the number of the page identifiers, or can be performed in a balanced manner according to software and hardware of each client.

It should be noted that the page identifiers may not be pre-allocated, but all the page identifiers are sequentially sorted into a table, and after each client completes the test task for one page, the current first untested page identifier in the table is automatically selected to execute the page test task. The load balancing task getting mode can avoid uneven task completion speed caused by hardware and software difference between clients, and improve the overall testing efficiency of target application.

In another implementation, multiple types of boundary values may be allocated to multiple clients, and an association table between the device identifier of each client and the allocated boundary value type is stored, so that each client performs a boundary value test of the corresponding type on each page. The invention realizes that the proxy server makes different modifications to the interface request response according to the different equipment identifications of the client to the interface mock of the single equipment. For example, a device may mock integer field, B device may mock string field, and both devices may test in parallel without affecting each other. Here, one device may execute one or more boundary value types, and if there are 18 clients, each client may execute only one boundary value type, and the proxy server may modify the response mock to only modify the corresponding type of boundary value. If 9 clients exist, each client can be allocated with two types of boundary values, and only the two types of boundary values are used for testing. Of course, the test efficiency can be maximized on the whole by performing balanced distribution according to the difference between hardware and software such as the memory and the network speed of the client.

Subsequently, in step S340, it is determined whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if yes, the fault information and the corresponding response value are automatically associated.

When the client runs the target application, the client reports the interface request and the response in the running process to the service server and attaches an equipment identifier, application version information, a software package name, running time and a unique task identifier. Once a fault occurs in the operation process, abnormal stack information of the fault is automatically captured and uploaded to the service server. Therefore, the fault information uploaded by the client may include a unique task identifier, a service interface request, a corresponding response value, a device identifier, a network status, application version information, a software package name, exception stack information, and the like when a fault occurs, and the fault information may be reported in a JSON format. An example of a piece of fault information is shown below:

the message format reported to the message queue contains the unique task identifier of the task, so that the fault information can be automatically associated with the response values such as the corresponding modified hierarchy, the boundary value type and the like in the message format through the unique task identifier. Specifically, the message queue may be consumed, the unique task identifiers may be taken out one by one, and whether corresponding fault information exists or not may be queried according to the unique task identifiers. If so, judging that the boundary value of the operation is abnormal, automatically associating the fault information with the corresponding message queue information, and sending the fault information to a tester. The notification messages it sends are as follows:

app packet 637 boundary test packet for automobile quoted price

App version number V6.2.1

Run time 2019-07-17

B27214cba92b11e99bd090b11c30468d task-unique id

And alarm content, wherein 1 abnormal log exists when the price and the android execute the cardDetailActivity boundary test. Details of Crash are described in http:// athm. Cn/QpnQKnc

Event type crash

String boundary value

Interface response set see http:// athm. Cn/QpnQKnc

FIG. 5 illustrates a flow diagram of an application testing method 500, suitable for execution in a proxy server, such as proxy server 120, which is primarily decoupled from application boundary testing, according to another embodiment of the invention.

As shown in fig. 5, the method 500 begins at step S510. In step S510, the data in the message queue is consumed, and the unique task identifier, the boundary value, the mock hierarchy, the page, the device identifier, and the time are obtained from the queue.

Subsequently, in step S520, it is queried whether there is corresponding fault information according to the globally unique task identifier (the fault information generated in the boundary test process is reported to the service server).

If the server side inquires the corresponding fault information, in step S530, it is determined that the boundary value corresponding to the current operation has a problem at the application side, and the request and the mock post-response data in the current operation process are inquired according to the unique task identifier executed at this time. Then, in step S540, the operation record is set as failure, and the queried result, the boundary test value and the page identifier taken out from the queue are sent to relevant personnel, such as a tester and a corresponding developer).

If the corresponding fault information is not queried, in step S550, it is determined that the boundary value corresponding to the operation does not have a problem at the application end, and the operation record is set to be successful.

In summary, the method 500 mainly determines whether the application is robust by detecting whether there is crash information during the running period, and automatically associates the interface data running at that time according to the unique task identifier to find out which boundary value the application has a problem.

FIG. 6 illustrates a flow diagram of an application testing method 600 according to yet another embodiment of the invention, the method 600 may be performed in the application testing system 100. As shown in fig. 6, the method 600 begins at step S610.

In step S610, the client sends a page start request for a certain page in the target application to the service server through the proxy server, where the page start request includes a unique task identifier and one or more interface requests for this start.

Subsequently, in step S620, the proxy server intercepts response contents of each interface request returned by the service server, where each response content includes multiple fields, and the multiple fields have a hierarchical relationship.

Subsequently, in step S630, the proxy server uniformly modifies the same layer field in each response content to a certain type of boundary value according to a predetermined rule, and returns the modified response value to the client.

Then, in step S640, the client runs the response value, and automatically reports current fault information to the service server when a fault occurs, where the fault information includes a current unique task identifier.

Subsequently, in step S650, the proxy server determines whether the unique task identifier has corresponding fault information from the reported record, and if so, automatically associates the fault information with a corresponding response value.

Fig. 7 shows a block diagram of an application testing apparatus 700 according to an embodiment of the present invention, and the apparatus 700 may reside in the proxy server 120. As shown in fig. 7, the apparatus 700 includes: a request receiving module 710, a response intercepting module 720, a response modifying module 730, and a failure analysis module 740.

The request receiving module 710 receives a page start request for a certain page in a target application sent by a client, where the page start request includes a unique task identifier and one or more interface requests of this start. The request receiving module 710 may perform processing corresponding to the processing described above in step S310, and a detailed description thereof will not be repeated.

The response intercepting module 720 intercepts response contents of the interface requests, each response content including a plurality of fields having a hierarchical relationship. The response intercepting module 720 may perform processing corresponding to the processing described above in step S320, which is not described herein again.

The response modification module 730 uniformly modifies the same layer of fields in each response content into a certain type of boundary values according to a predetermined rule, and returns the modified response values to the client for operation. The response modification module 730 may perform processing corresponding to the processing described above in step S330, which is not described herein again.

The failure analysis module 740 determines whether the unique task identifier has corresponding failure information from the failure information reported by the client, and if so, automatically associates the failure information with the corresponding response value. The failure analysis module 740 may perform processing corresponding to the processing described above in step S340, and details thereof are not repeated here.

According to an embodiment of the present invention, the apparatus 700 may further include a reporting module and an allocating module (neither shown in the figure). The reporting module can send the only task identifier, the page identifier, the modified hierarchy and boundary value type, the equipment identifier and the operation starting time which are started at this time to the message queue, and at this time, the page testing task is considered to be completed once. The allocation module may allocate a plurality of page identifiers to a plurality of clients under one condition, and store an association table between the device identifier of the client and the allocated page identifier, so that each client executes a page test task corresponding to each page identifier. The allocation module may allocate a plurality of types of boundary values to a plurality of clients, and store an association table of the device identifier of each client and the allocated boundary value type, so that each client performs a boundary value test of a corresponding type on each page.

According to another embodiment of the present invention, the apparatus 700 may further include a restart module, adapted to clean the test environment of the target application after each page test task is completed, and perform page test tasks of other page identifiers or other levels or other boundary value types until page test tasks of all boundary value types of all levels under all page identifiers are completed.

According to the technical scheme of the invention, a method and a system for fully-automatic application robustness testing with high coverage rate and low maintenance cost are established. By intercepting the network request of the client, the interface response is modified into an irregular data return value according to a preset rule, and the blast information of the application terminal is automatically collected and associated with mock data, so that the code with defects is positioned. The 18 types of boundary values set by the method cover the common boundary values of the application end, and the type analysis errors possibly existing in the application end are exposed. The interface hierarchical mock can fully expose the logic of the interface processed by the application end each time, and avoid that the inner layer logic error can not be exposed due to the outer layer processing logic error. The high coverage rate of the boundary test is ensured through two measures of boundary value multi-type and hierarchical mock. In addition, the invention sets the schema for each page to be tested, and each page is independently started through the schema without operating according to the page path of the application, thereby ensuring that each page can be directly started and improving the stability of the boundary test.

A6 the method according to any of A1-A5, wherein the page activation request is sent in a schema manner, and the field of each response content is divided into at most ten levels. The method as recited in A5, wherein the proxy server is communicatively connected to a plurality of clients, each client including the target application, the method includes the steps of: and distributing the plurality of page identifications to the plurality of clients, and storing an association relation table of the device identifications of the clients and the distributed page identifications so that each client can execute the page test task corresponding to each page identification. A8, the method of any one of A1-A6, further comprising the steps of: and distributing the multiple types of the boundary values to multiple clients, and storing an association relation table of the equipment identification of each client and the distributed boundary value type so that each client can execute boundary value test of the corresponding type on each page.

The method according to any one of the claims A9 to A8, wherein the interface requests include service class interface requests and non-service class interface requests, and the proxy server only modifies response contents of the service class interface requests. The method according to a10 and A9, wherein the proxy server further stores a domain name white list of the service class interface, and the method further includes: and acquiring a domain name value in a request header of each interface request, and identifying the service type interface request according to the domain name white list. The method as in A9, wherein the traffic interface is logically related to an application interface, and the non-traffic interface includes a buried point interface and an advertisement exposure request interface. A12, the method of any one of A1-A11, further comprising the steps of: and constructing a special boundary test packet, and sending the special boundary test packet to a client for installation, wherein the special boundary test packet supports a mock function and an interface log uploading function.

A13, as in any one of the methods A1 to a12, where the fault information reported by the client in the running process includes a unique task identifier when a fault occurs, a service interface request, and a corresponding modified response value. A14, the method as in a13, wherein the failure information is reported in JSON format, and further includes at least one of device identifier, network status, application version information, package name, and exception stack information. A15, the method of any one of A1-a14, further comprising the steps of: consuming the message queue, taking out the unique task identifiers one by one, and inquiring whether corresponding fault information exists according to the unique task identifiers; if so, judging that the boundary value of the operation is abnormal, automatically associating the fault information with the corresponding message queue information, and sending the fault information to a tester.

The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U.S. disks, floppy disks, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.

In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to execute the application testing method of the present invention according to instructions in the program code stored in the memory.

By way of example, and not limitation, readable media may comprise readable storage media and communication media. Readable storage media store information such as computer readable instructions, data structures, program modules or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.

In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with examples of this invention. The required structure for constructing such a system will be apparent from the description above. Moreover, the present invention is not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into multiple sub-modules.

Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore, may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Furthermore, some of the described embodiments are described herein as a method or combination of method elements that can be performed by a processor of a computer system or by other means of performing the described functions. A processor with the necessary instructions for carrying out the method or the method elements thus forms a device for carrying out the method or the method elements. Further, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is used to implement the functions performed by the elements for the purpose of carrying out the invention.

As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of this description, will appreciate that other embodiments can be devised which do not depart from the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the appended claims. The present invention has been disclosed by way of illustration and not limitation with respect to the scope of the invention, which is defined by the appended claims.

Claims

1. An application testing method adapted to be executed in a proxy server, the method comprising the steps of:

receiving a page starting request for a certain page in a target application, which is sent by a client, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time;

intercepting response contents of each interface request, wherein each response content comprises a plurality of fields, and the fields have a hierarchical relationship;

uniformly modifying the same layer of fields in each response content into a certain type of boundary value according to a preset rule, returning the modified response value to the client for running, and then sending the unique task identifier, the page identifier, the modified level and boundary value type, the equipment identifier and the running starting time which are started at this time into a message queue, wherein at this moment, a page testing task is considered to be completed; and

judging whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if so, automatically associating the fault information with a corresponding response value;

wherein the predetermined rule comprises: modifying the field of each response content layer by layer, modifying the same layer of field of all response contents each time, modifying the layer of field into the same type of boundary value each time, and modifying the next layer of field of all response contents after traversing all types of boundary values of one layer of field.

2. The method of claim 1, wherein the type of boundary value comprises at least one of:

random string, super-long string, special string, integer, single precision floating point, double precision floating point, time, array, boolean, url, mailbox, IP, picture, expression, null array, null object, null, undefined.

3. The method of claim 1, wherein the target application includes a plurality of page identifications, the method further comprising the steps of:

and after the page test task is completed once, cleaning the test environment of the target application, and performing page test tasks of other page identifications or other levels or other boundary value types until the page test tasks of all boundary value types of all levels under all page identifications are completed.

4. The method of claim 1, wherein the page activation request is transmitted by means of a schema, and a field of each response content is divided into a maximum of ten levels.

5. The method of claim 3, wherein the proxy server is communicatively coupled to a plurality of clients, each client containing the target application, the method comprising the steps of:

and distributing the plurality of page identifications to the plurality of clients, and storing an association relation table of the device identifications of the clients and the distributed page identifications so that each client can execute the page test task corresponding to each page identification.

6. The method of claim 1, further comprising the steps of:

and distributing the multiple types of the boundary values to multiple clients, and storing an association relation table of the equipment identification of each client and the distributed boundary value type so that each client can execute boundary value test of the corresponding type on each page.

7. The method of claim 1, wherein the interface requests include traffic class interface requests and non-traffic class interface requests, and the proxy server modifies only response content of traffic class interface requests.

8. The method of claim 7, wherein the proxy server further stores a domain name white list of traffic class interfaces, the method further comprising the steps of:

and acquiring a domain name value in a request header of each interface request, and identifying the service type interface request according to the domain name white list.

9. The method of claim 7, wherein the traffic class interface is logically associated with an application interface and the non-traffic class interface comprises a buried point interface and an advertisement exposure request interface.

10. The method of claim 1, further comprising the steps of:

and constructing a special boundary test packet, and sending the special boundary test packet to a client for installation, wherein the special boundary test packet supports a mock function and an interface log uploading function.

11. The method of claim 1, wherein the failure information reported by the client during the operation process includes a unique task identifier, a service interface request and a corresponding modified response value when a failure occurs.

12. The method of claim 11, wherein the fault information is reported in JSON format, further comprising at least one of device identification, network condition, application version information, package name, and exception stack information.

13. The method of claim 1, further comprising the steps of:

consuming the message queue, taking out the unique task identifiers one by one, and inquiring whether corresponding fault information exists according to the unique task identifiers;

if so, judging that the boundary value of the operation is abnormal, automatically associating the fault information with the corresponding message queue information, and sending the fault information to a tester.

14. An application testing method comprising:

a client sends a page starting request for a certain page in a target application to a service server through a proxy server, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time;

the method comprises the steps that a proxy server intercepts response contents of interface requests returned by a service server, wherein each response content comprises a plurality of fields, and the fields have a hierarchical relationship;

the proxy server uniformly modifies the same layer of fields in each response content into a certain type of boundary value according to a preset rule, returns the modified response value to the client, and then sends the unique task identifier, the page identifier, the modified layer and boundary value type, the equipment identifier and the operation starting time which are started at this time into a message queue, wherein the page testing task is considered to be completed once at this time;

the client operates the response value and automatically reports the current fault information when a fault occurs, wherein the fault information comprises the current unique task identifier; and

the proxy server judges whether the unique task identifier has corresponding fault information from the reported records, and if so, automatically associates the fault information with the corresponding response value;

15. An application testing apparatus adapted to reside in a proxy server, the apparatus comprising:

the request receiving module is suitable for receiving a page starting request for a certain page in the target application, which is sent by a client, wherein the page starting request comprises a unique task identifier and one or more interface requests which are started at this time;

the response intercepting module is suitable for identifying business interface requests in the one or more interface requests and intercepting response contents of the business interface requests, wherein each response content comprises a plurality of layers of fields;

the response modification module is suitable for uniformly modifying the same layer of fields in each response content into a certain type of boundary value according to a preset rule, returning the modified response value to the client for operation, and then sending the unique task identifier, the page identifier, the modified level and boundary value type, the equipment identifier and the operation starting time which are started at this time into the message queue, wherein a page test task is considered to be completed at this time; and

the fault analysis module is suitable for judging whether the unique task identifier has corresponding fault information from the fault information reported by the client, and if so, automatically associating the fault information with a corresponding response value;

16. A proxy server, comprising:

one or more processors;

a memory; and

one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs, when executed by the processors, implementing the steps of the method of any of claims 1-13.

17. A readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a proxy server, perform the steps of the method of any of claims 1-13.

18. An application testing system comprising:

one or more clients, a business server, and the proxy server of claim 16;

the proxy server is respectively in communication connection with the service server and the one or more clients, and is suitable for receiving page starting requests sent by the one or more clients and intercepting response contents returned by the service server for modification.