CN112860572A

CN112860572A - Cloud testing method, device, system, medium and electronic equipment of mobile terminal

Info

Publication number: CN112860572A
Application number: CN202110269315.6A
Authority: CN
Inventors: 仲林霖
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Hangzhou Netease Shuzhifan Technology Co ltd
Priority date: 2021-03-12
Filing date: 2021-03-12
Publication date: 2021-05-28
Anticipated expiration: 2041-03-12
Also published as: CN112860572B

Abstract

The utility model provides a cloud test method, device, system, medium and electronic equipment of mobile terminal, relating to the technical field of cloud test, comprising: receiving an access request to a mobile terminal sent by a client; responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client; receiving an operation instruction sent by a client, and converting the operation instruction according to the system type of an operation system of the mobile terminal to obtain a target instruction; and sending the target instruction to the mobile terminal based on the preset interface so that the mobile terminal executes the target instruction in the first current display interface to realize a cloud test task for the mobile terminal. The cloud test method and the cloud test system realize the cloud test of the mobile terminal with two different types of systems through the same test scheme.

Description

Cloud testing method, device, system, medium and electronic equipment of mobile terminal

Technical Field

Embodiments of the present disclosure relate to the field of cloud testing technologies, and in particular, to a cloud testing method for a mobile terminal, a cloud testing apparatus for a mobile terminal, a cloud testing system for a mobile terminal, a computer-readable storage medium, and an electronic device.

Background

This section is intended to provide a background or context to the embodiments of the disclosure recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.

In some existing solutions for cloud testing (remote access or desktop control) of a mobile terminal with an Android system and a mobile terminal with an iOS system, they are implemented by two different methods, respectively; in addition, in a specific test process, the adopted interactive mode and the integration mode are different.

The existing technical scheme can not realize the cloud test of the mobile terminal with two different types of systems through the same test scheme.

Disclosure of Invention

However, in some technologies, it is not possible to perform cloud testing on mobile terminals having two different types of systems through the same testing scheme. Therefore, an improved cloud test method for a mobile terminal is highly needed to implement cloud test on mobile terminals with two different types of systems through the same test scheme, so as to improve test efficiency.

In this context, embodiments of the present disclosure desirably provide a cloud test method of a mobile terminal, a cloud test apparatus of a mobile terminal, a cloud test system of a mobile terminal, a computer-readable storage medium, and an electronic device.

According to one aspect of the present disclosure, a cloud testing method of a mobile terminal, configured in a proxy server, includes:

receiving an access request to a mobile terminal sent by a client;

responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client;

receiving an operation instruction sent by the client, and converting the operation instruction according to the system type to which the operation system of the mobile terminal belongs to obtain a target instruction; wherein the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface;

and sending the target instruction to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, and a cloud test task of the mobile terminal is realized.

In an exemplary embodiment of the present disclosure, the cloud testing method of the mobile terminal further includes:

receiving a target display interface uploaded by the mobile terminal after the target instruction takes effect, and extracting target image features included in the target display interface based on a preset image recognition algorithm;

analyzing the difference of the target image characteristics, and judging whether the target display interfaces are consistent according to the difference analysis result;

and when the target display interface is determined to be inconsistent, determining the reason for causing the difference of the target display interface according to the target image characteristics.

In an exemplary embodiment of the present disclosure, determining a cause of dissimilarity of the target display interface according to the target image feature includes:

extracting a popup window arranged on the upper layer of the target display interface based on the preset image recognition algorithm, and recognizing the packet name to which the popup window belongs;

and determining the category of the popup window according to the packet name recognition result, and determining the reason causing the difference of the target display interface according to the category of the popup window.

In an exemplary embodiment of the present disclosure, if the category of the popup is a system popup, determining that a cause of the dissimilarity of the target display interface is: a system popup is arranged on the upper layer of the target display interface, so that the target instruction cannot be executed in the current display interface;

if the type of the popup is application popup, the reason for causing the difference of the target display interface is that the application popup floats on the target display interface, so that the target instruction cannot act on the current display interface.

generating a first closing instruction corresponding to the system popup according to a first instruction generation rule corresponding to the system popup;

and sending the first closing instruction to the mobile terminal so that the mobile terminal executes the first closing instruction to close the system popup.

identifying the popup window of the application program by using the preset image identification algorithm to obtain name information of the application program;

matching the authority of the target application program corresponding to the name information in the mobile terminal based on the preset interface;

generating a second closing instruction corresponding to the application program popup according to the authority of the target application program, the information displayed in the application program popup and a second instruction generation rule corresponding to the application program popup;

and sending the second closing instruction to the mobile terminal so that the mobile terminal executes the second closing instruction to close the application popup window.

In an exemplary embodiment of the present disclosure, determining a cause of dissimilarity of the target display interface according to the target image feature further includes:

extracting the current image characteristics of the first current display interface based on the preset image recognition algorithm, and comparing the current image characteristics with the target image characteristics;

and determining reasons causing the difference of the target display interface according to the comparison result of the current image characteristics and the target image characteristics.

In an exemplary embodiment of the disclosure, if the comparison result of the current image feature and the target image feature is the same, it is determined that the reason causing the difference of the target display interface is: and the instruction coordinate included in the target instruction is inconsistent with the actual coordinate of the mobile terminal.

and performing equal-proportion shifting or line changing on the instruction coordinate according to the actual length and the actual width of the display interface of the mobile terminal, and updating the target instruction according to the shifted or line-changed instruction coordinate.

In an exemplary embodiment of the present disclosure, the system class includes an android operating system or an iOS operating system;

wherein, according to the system type to which the operating system of the client belongs, converting the operating instruction to obtain a target instruction comprises:

analyzing the operation instruction to obtain the gesture operation included in the operation instruction, and an original abscissa and an original ordinate of the gesture operation;

converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the android operating system to obtain a first target instruction required by a mobile terminal with the android operating system; or

And converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the iOS operating system to obtain a second target instruction required by the mobile terminal with the iOS operating system.

In an exemplary embodiment of the present disclosure, converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the android operating system to obtain a first target instruction required by a mobile terminal having the android operating system includes:

extracting first character information included in the gesture operation as target name information of the gesture operation;

converting the original abscissa and the original ordinate according to a preset coordinate conversion principle to obtain a target abscissa and a target ordinate;

and splicing the target name information, the target abscissa and the target ordinate corresponding to the target name information to obtain a first sub-target instruction corresponding to each gesture operation, and generating the first target instruction according to the first sub-target instruction.

According to an aspect of the present disclosure, there is provided a cloud testing apparatus of a mobile terminal, configured in a proxy server, the cloud testing apparatus of the mobile terminal including:

the request receiving module is used for receiving an access request to the mobile terminal, which is sent by the client;

the picture acquisition module is used for responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client;

the instruction conversion module is used for receiving the operation instruction sent by the client and converting the operation instruction according to the system type of the operation system of the mobile terminal to obtain a target instruction; wherein the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface;

and the cloud testing module is used for sending the target instruction to the mobile terminal based on the preset interface so that the mobile terminal executes the target instruction in the first current display interface to realize a cloud testing task for the mobile terminal.

In an exemplary embodiment of the present disclosure, the cloud testing apparatus of the mobile terminal further includes:

the first feature extraction module is used for receiving a target display interface uploaded by the mobile terminal after the target instruction takes effect, and extracting target image features included in the target display interface based on a preset image recognition algorithm;

the first analysis module is used for analyzing the difference of the target image characteristics and judging whether the target display interfaces are consistent according to the difference analysis result;

and the difference reason determining module is used for determining the reason causing the difference of the target display interface according to the target image characteristics when the target display interface is determined to be inconsistent.

the first instruction generation module is used for generating a first closing instruction corresponding to the system popup according to a first instruction generation rule corresponding to the system popup;

and the first instruction sending module is used for sending the first closing instruction to the mobile terminal so that the mobile terminal executes the first closing instruction to close the system popup window.

the popup identification module is used for identifying the popup of the application program by utilizing the preset image identification algorithm to obtain the name information of the application program;

the permission matching module is used for matching the permission of the target application program corresponding to the name information in the mobile terminal based on the preset interface;

the second instruction generation module is used for generating a second closing instruction corresponding to the application program popup according to the authority of the target application program, the information displayed in the application program popup and a second instruction generation rule corresponding to the application program popup;

and the second instruction sending module is used for sending the second closing instruction to the mobile terminal so that the mobile terminal executes the second closing instruction to close the application popup window.

and the target instruction updating module is used for shifting or line-changing the instruction coordinate in equal proportion according to the actual length and the actual width of the display interface of the mobile terminal and updating the target instruction according to the shifted or line-changed instruction coordinate.

According to an aspect of the present disclosure, there is provided a cloud test system of a mobile terminal, including:

the client is used for generating an access request at the mobile terminal and sending the access request to the server to be served; and generating an operation instruction;

the proxy server is connected with the client network and used for realizing the cloud testing method of the mobile terminal;

and the mobile terminal is connected with the proxy server based on a preset interface and is used for executing the target instruction on a first current display interface.

According to an aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a cloud testing method of a mobile terminal as described in any one of the above.

According to an aspect of the present disclosure, there is provided an electronic device including:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to execute any one of the above cloud testing methods of the mobile terminal via execution of the executable instructions.

According to the cloud test method and the cloud test device of the mobile terminal, an access request sent by a client to the mobile terminal can be received; then responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client; receiving an operation instruction sent by the client, and converting the operation instruction according to the system type of an operation system of the mobile terminal to obtain a target instruction; and finally, the target instruction is sent to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, a cloud test task for the mobile terminal is realized, and the mobile terminal with different system types does not need to be subjected to cloud test through different test schemes, so that the time cost caused by the need of deploying different test schemes is remarkably reduced, the problem of low test efficiency caused by the need of replacing the test schemes when the mobile terminal with different system types is subjected to cloud test is solved, and better experience is brought to a user.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

fig. 1 schematically shows a flowchart of a cloud testing method of a mobile terminal according to an example embodiment of the present disclosure;

fig. 2 schematically illustrates a block diagram of a cloud testing system of a mobile terminal according to an example embodiment of the present disclosure;

fig. 3 schematically illustrates a flowchart of another cloud testing method of a mobile terminal according to an example embodiment of the present disclosure;

FIG. 4 schematically illustrates a flow chart of a method of generating a second close instruction according to an example embodiment of the present disclosure;

FIG. 5 schematically illustrates a flow chart of a method of deploying a proxy server according to an example embodiment of the present disclosure;

FIG. 6 schematically illustrates a flow chart of a first display interface of a mobile terminal according to an example embodiment of the present disclosure;

fig. 7 schematically illustrates a flowchart of another cloud testing method of a mobile terminal according to an example embodiment of the present disclosure;

fig. 8 is a diagram schematically illustrating an application scenario example of a cloud testing method for a mobile terminal according to an exemplary embodiment of the present disclosure;

fig. 9 schematically illustrates a block diagram of a cloud testing apparatus of a mobile terminal according to an example embodiment of the present disclosure;

fig. 10 schematically illustrates a computer-readable storage medium for storing the cloud test method of the mobile terminal according to an exemplary embodiment of the present disclosure;

fig. 11 schematically illustrates an electronic device for implementing the cloud testing method of the mobile terminal according to an exemplary embodiment of the present disclosure.

In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

Detailed Description

The principles and spirit of the present disclosure will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely for the purpose of enabling those skilled in the art to better understand and to practice the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be embodied as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

According to the embodiment of the disclosure, a cloud testing method, a device, a system, a computer readable storage medium and an electronic device of a mobile terminal are provided.

In this document, any number of elements in the drawings is by way of example and not by way of limitation, and any nomenclature is used solely for differentiation and not by way of limitation.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

Summary of The Invention

The present disclosure finds that, in some schemes of performing remote control on an Android terminal (one or more Android or iOS devices may be controlled through a world wide web or a local area network to perform clicking, sliding, dragging, and other gesture operations) or performing remote access (a mobile phone screen of a remote mobile phone may be browsed and controlled through a world wide web or a local area network at a local program interface or a web page), a service running in the mobile phone may be accessed from an external web or an application through a network or a USB by running a service based on a TCP/IP protocol in the Android mobile phone, so that a connection may be established. Through the connection, the outside can acquire the picture of the mobile phone in real time, and send gesture operation (including clicking, sliding, dragging and the like) to service execution in the mobile phone from the outside.

In some schemes for remote control or access of the iOS terminal, the remote control or access can be implemented based on open-source WebdriverAgent. The WebdriverAgent is a service running in the iOS device, and based on an http protocol, the service running in the mobile phone can be accessed from an external webpage or an application program through a network or a USB mode, so that connection can be established. Through the link, the outside can acquire the picture of the mobile phone in real time, and sends gesture operations (including clicking, sliding, dragging and the like) to the WebdriverAgent service running in the mobile phone from the outside.

However, in the above-mentioned embodiments, the technologies for remotely accessing and controlling the desktop for the Android and iOS systems are different, and the interactive mode and the integration mode are also different. The existing technical scheme does not solve the problem of access service universality.

Based on this, the disclosed example embodiment provides a cloud testing method for a mobile terminal, on one hand, by receiving an access request for the mobile terminal sent by a client; then responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client; receiving an operation instruction sent by the client, and converting the operation instruction according to the system type of an operation system of the mobile terminal to obtain a target instruction; the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface; finally, the target instruction is sent to the mobile terminal based on a preset interface, so that the mobile terminal executes the target instruction in the first current display interface, and a cloud test task of the mobile terminal is realized; the cloud testing method of the mobile terminal can be operated at the proxy server end, and can convert operation execution according to the system type to which the operating system of the mobile terminal belongs, so that the problem that the cloud testing of the mobile terminals of different system types cannot be carried out through the same testing scheme in the prior art can be solved, the unified testing of the mobile terminals of different system types is realized, and the problem of the universality of access service is solved; on the other hand, the exemplary embodiment of the present disclosure can also convert the operation instruction to obtain the target instruction according to the system type to which the operation system of the mobile terminal belongs; and then, the target instruction is sent to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, and a cloud test task for the mobile terminal is realized, thereby simplifying the cloud test flow of the mobile terminal and improving the test efficiency of the cloud test.

Having described the general principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

Exemplary method

The cloud testing method of the mobile terminal is provided firstly in the disclosed example embodiment, and the method can be operated in a proxy server, a server cluster or a cloud server and the like; of course, those skilled in the art may also operate the method of the present disclosure on other platforms as needed, which is not particularly limited in the exemplary embodiment. Referring to fig. 1, the cloud testing method of the mobile terminal may include the steps of:

step S110, receiving an access request sent by a client to a mobile terminal;

s120, responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client;

s130, receiving an operation instruction sent by the client, and converting the operation instruction according to the system type of an operation system of the mobile terminal to obtain a target instruction; wherein the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface;

step S140, the target instruction is sent to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, and a cloud test task of the mobile terminal is achieved.

In the cloud test method of the mobile terminal, an access request for the mobile terminal sent by a client is received; then responding to the access request, acquiring a first current display interface of the mobile terminal in real time based on a preset interface, and sending the first current display interface to the client so as to generate and display a second current display interface corresponding to the first current display interface on the client; receiving an operation instruction sent by the client, and converting the operation instruction according to the system type of an operation system of the mobile terminal to obtain a target instruction; and finally, the target instruction is sent to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, a cloud test task for the mobile terminal is realized, and the mobile terminal with different system types does not need to be subjected to cloud test through different test schemes, so that the time cost caused by the need of deploying different test schemes is remarkably reduced, the problem of low test efficiency caused by the need of replacing the test schemes when the mobile terminal with different system types is subjected to cloud test is solved, and better experience is brought to a user.

Hereinafter, a cloud test method of a mobile terminal according to an exemplary embodiment of the present disclosure will be explained and explained with reference to the drawings.

First, basic services of an Android end and basic services of an iOS end are introduced.

On one hand, the minicap and/or minibutton supported by the Android end are developed and sourced by an openstf team, the minicap and the minibutton are deployed on Android equipment and started to respectively start services, and the communication protocol is a TCP protocol. And respectively realizing quick real-time screenshot of the Android equipment picture by calling a C language interface at the bottom layer of the Android system, receiving control data in real time, and calling a system interface to execute in real time to realize real-time control of the Android equipment.

On the other hand, the WebdriverAgent supported by the iOS end is firstly developed and sourced by Facebook, the WebdriverAgent is installed on the iOS equipment and started, a service is started on the mobile phone, the communication protocol is http protocol, the real-time picture of the iOS mobile phone can be obtained through the service, the control data can also be sent to the service, and the WebdriverAgent receives, analyzes and executes the control data.

Next, the purpose and specific scenes of the invention of the present disclosure will be explained and explained.

The problems mainly solved by the method are as follows: the complete remote control of the Android and/or iOS can realize the remote test of the mobile terminal for the client (the client may include a webpage and/or an application program), regardless of whether the mobile terminal connected with the proxy server is a mobile terminal with an Android system or a mobile terminal with an iOS system. The specific reason is as follows: according to the scheme described herein, an agent is designed and implemented for the basic services at both ends of the Android and iOS, the services at both ends are managed in a centralized manner, and communication data protocols suitable for the Android end and the iOS end are converted respectively.

Further, the cloud test system of the mobile terminal of the exemplary embodiment of the present disclosure is explained and illustrated.

Referring to fig. 2, the cloud test system of the mobile terminal may include a client 210, a proxy server 220, and a mobile terminal 230. Wherein:

the client 210 may be configured to generate an access request at the mobile terminal and send the access request to the standby server; and generating an operation instruction; the proxy service 220 is in communication connection with the client, and is used for implementing the cloud testing method of the mobile terminal described in the exemplary embodiment of the present disclosure; and the mobile terminal 230 is connected with the proxy server based on a preset interface and is used for executing the target instruction on the first current display interface.

The preset interface can be a USB interface, which not only can realize the connection between the mobile terminal and the proxy server, but also the proxy server can supply power to the mobile terminal through the USB interface, so that the mobile terminal can be in a standby state for a long time; moreover, the mobile terminal is a plurality of mobile terminals with different system types, and the system type may include an Android system, an iOS system, a Linux system, or a damming system, which is not particularly limited in this example.

The Android and iOS terminals respectively run respective basic services, the Server terminal runs an agent program, and the Client terminal runs a webpage or other application programs; meanwhile, the Server side carries out data communication with the Android and iOS terminals through the USB, and the data interaction of basic services at the Android and iOS terminals can be compatible at the same time through the agent program operated by the Server side.

The Client accesses the service provided by the proxy program of the Server through a TCP/IP protocol, and the Client can acquire real-time pictures of the Android and the iOS in real time through the proxy program of the Server. The method comprises the steps that gesture operation data generated by a Client side are sent to basic services of an Android side and an iOS side through a proxy program of a Server side, before the gesture operation data of the Client side are forwarded, the proxy program of the Server side converts data formats of the gesture operation data respectively aiming at a currently used mobile phone operating system, and then the gesture operation data capable of being executed by each system are sent to a mobile terminal to be executed.

Next, steps S110 to S140 will be explained and explained with reference to fig. 2.

In step S110, an access request for the mobile terminal sent by the client is received.

Specifically, when the mobile terminal needs to be subjected to cloud testing, an access request can be generated through a webpage of the client or an application program, and then the access request is sent to the proxy server.

In step S120, in response to the access request, a first current display interface of the mobile terminal is obtained in real time based on a preset interface, and the first current display interface is sent to the client, so that a second current display interface corresponding to the first current display interface is generated and displayed on the client.

Specifically, after the proxy server receives the access request, the first current display interface of each mobile terminal may be obtained in real time based on a USB interface between the proxy server and the mobile terminal (certainly, other interfaces that may provide a power supply or implement communication connection, such as a type-c interface, which is not particularly limited in this example), and then the first current display interface is sent to the client.

In step S130, receiving an operation instruction sent by the client, and converting the operation instruction according to a system type to which an operating system of the mobile terminal belongs to obtain a target instruction; wherein the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface.

Specifically, the user may input a corresponding gesture operation according to the UI control included in the second current display interface, where the gesture operation may include operations such as clicking, sliding, and the like, and may also include other operations, which is not particularly limited in this example; after receiving the gesture operation, the client may generate an operation instruction according to the gesture operation, and send the operation instruction to the proxy server.

Further, after the proxy server receives the operation instruction, in order to enable the operation instruction to be applicable to mobile terminals of different system types, the operation instruction needs to be converted, and here, an android operating system and/or an iOS operating system are taken as an example, and a specific conversion process is explained and explained. Specifically, converting the operation instruction to obtain a target instruction according to the system type to which the operating system of the client belongs may include: firstly, analyzing the operation instruction to obtain the gesture operation included in the operation instruction, and an original abscissa and an original ordinate of the gesture operation; secondly, converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the android operating system to obtain a first target instruction required by the mobile terminal with the android operating system; or converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the iOS operating system to obtain a second target instruction required by the mobile terminal with the iOS operating system.

Converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the android operating system to obtain a first target instruction required by the mobile terminal with the android operating system, where the converting may include: firstly, extracting first character information included in the gesture operation as target name information of the gesture operation; secondly, converting the original abscissa and the original ordinate according to a preset coordinate conversion principle to obtain a target abscissa and a target ordinate; and finally, splicing the target name information, a target abscissa and a target ordinate corresponding to the target name information to obtain a first sub-target instruction corresponding to each gesture operation, and generating the first target instruction according to the first sub-target instruction.

Further, taking the example that the operation instruction of the client (client) includes click and slide data, the specific conversion process is further explained and explained.

click and slide operation data generated by the client are shown as follows:

{

"type":"down",

"x":0.5862068965517241,

"y":0.5826681870011402,

"rotation":0

}、

{

"type":"move",

"x":0.5862068965517241,

"y":0.5826681870011402,

"rotation":0

}、

{

"type":"up",

"x":0.5862068965517241,

"y":0.5826681870011402

}

in the above example, the types of actions included in the type field include down, move, and up, which respectively represent three actions of pressing, moving, and lifting.

x field: the abscissa generated by the motion is proportional to the width of the handset.

y field: the ratio of the ordinate produced by the action to the height of the handset.

When the proxy server acts on the Android client, the data sent to the Android system to execute the operation are shown as follows:

d 500 1000\n

m 500 1000\n

u\n

the first letter of each row respectively represents down, move and up, and the latter two numbers respectively represent horizontal and vertical coordinates of the action, and the actual coordinate values are converted according to the mobile phone size in proportion.

When the proxy server proxies the iOS client, the data sent to the iOS system for execution is presented as follows:

{

"actions":[

{

"options":{

"x":500,

"y":1000

},

"action":"press"

},

{

"options":{

"x":500,

"y":1000

},

"action":"moveTo"

},

{

"options":{

},

"action":"release"

}

]

}

wherein, actions field, represents the set of all actions; the actions in the set are ordered, the action field of each action represents the type of the action, the three types of press (press), moveTo (move) and release (lift), and the options field represents the coordinates of the action.

In step S140, the target instruction is sent to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface, thereby implementing a cloud test task for the mobile terminal.

Specifically, after the target instruction conversion is completed, the target instruction can be sent to the mobile terminal based on the USB interface, so that the mobile terminal can execute instructions such as pressing, moving, or lifting in the first current display interface, and further, a cloud test task is realized.

Further, in order to avoid a problem that a cloud test fails due to a fact that a coordinate position of a gesture operation included in a popup or target instruction in a first current display interface of the mobile terminal is not consistent with a coordinate position of the mobile terminal, an exemplary embodiment of the present disclosure further provides another cloud test method of the mobile terminal. Referring to fig. 3, the cloud testing method of the mobile terminal may further include the steps of:

step S310, receiving a target display interface uploaded by the mobile terminal after the target instruction takes effect, and extracting target image features included in the target display interface based on a preset image recognition algorithm.

Specifically, the image recognition algorithm may include a fast R-CNN, a SIFT algorithm, a SURF algorithm, and the like, which is not particularly limited in this example. In the process of extracting the target image features, the method can be implemented based on the basic principle of each algorithm, and is not described herein again.

Step S320, analyzing the difference of the target image features, and determining whether the target display interfaces are consistent according to the difference analysis result.

Specifically, after the target image features are obtained, differences of the target image features of the target display interfaces uploaded by a plurality of different mobile terminals (which may be mobile terminals of the same system type or mobile terminals of different system types, which is not particularly limited in this example) at the same time may be analyzed, and then whether the target display interfaces are consistent or not is determined according to a difference analysis result; that is, whether each target instruction executed successfully may be determined based on the diversity analysis result. It should be added that, in order to facilitate the analysis of the difference, the target display interfaces after successful execution may be the same by default, and therefore if the target image features are completely the same, there is no difference, the target display interfaces are the same; if the target image characteristics are different, the difference exists, and the target display interface is inconsistent.

Step S330, when the target display interface is determined to be inconsistent, determining the reason causing the difference of the target display interface according to the target image characteristics.

In the embodiment of the present invention, first, a popup window disposed on an upper layer of the target display interface is extracted based on the preset image recognition algorithm, and a package name to which the popup window belongs is recognized; secondly, determining the category of the popup window according to the packet name recognition result, and determining the reason causing the difference of the target display interface according to the category of the popup window. Wherein, if the category of the popup is a system popup, determining that the reason for causing the difference of the target display interface is: a system popup is arranged on the upper layer of the target display interface, so that the target instruction cannot be executed in the current display interface; if the type of the popup is application popup, the reason for causing the difference of the target display interface is that the application popup floats on the target display interface, so that the target instruction cannot act on the current display interface.

For example, the system popup may include a popup pushed by a system program carried by each mobile terminal, such as a software update popup, an electric quantity prompt popup, and the like; the application popup may include a popup pushed by an application in each mobile terminal, such as a wechat popup, an XX cloud music popup, and the like, which is not limited in this example.

Further, when the pop-up window is a system pop-up window, in order to implement a cloud test on the mobile terminal, the cloud test method of the mobile terminal further includes: generating a first closing instruction corresponding to the system popup according to a first instruction generation rule corresponding to the system popup; and sending the first closing instruction to the mobile terminal so that the mobile terminal executes the first closing instruction to close the system popup. For example, when the system popup is an electric quantity prompt popup, a first closing instruction may be generated to close the electric quantity prompt popup according to an operation to be executed in an actual operation process.

Further, when the pop-up window is an application pop-up window, in order to implement a cloud test on the mobile terminal, referring to fig. 4, the cloud test method for the mobile terminal may further include the following steps:

step S410, identifying the popup window of the application program by using the preset image identification algorithm to obtain name information of the application program;

step S420, matching the authority of the target application program corresponding to the name information in the mobile terminal based on the preset interface;

step S430, generating a second closing instruction corresponding to the application program popup according to the authority of the target application program, the information displayed in the application program popup and a second instruction generation rule corresponding to the application program popup;

step S440, sending the second closing instruction to the mobile terminal, so that the mobile terminal executes the second closing instruction to close the application popup.

Hereinafter, steps S410 to S440 will be explained and explained. Specifically, firstly, characters included in the pop-up window of the application program can be identified based on an image identification algorithm, and then corresponding name information is obtained (for example, the name information can be the royal glory, and the specific content included in the pop-up window is XX teammate upgrade), and after the name information is obtained, the authority corresponding to the glory of the royal can be matched from the mobile terminal based on the USB interface; and generating a corresponding second closing instruction according to the matched authority and the specific content so as to close the application popup. By the method, the problem of cloud test failure caused by the existence of the popup window can be avoided, and the success rate of the cloud test is further improved.

In another embodiment of the present example, determining the cause of the dissimilarity of the target display interface according to the target image feature further includes: firstly, extracting the current image characteristics of the first current display interface based on the preset image recognition algorithm, and comparing the current image characteristics with the target image characteristics; secondly, determining reasons causing difference of the target display interface according to the comparison result of the current image characteristics and the target image characteristics.

If the comparison result of the current image feature and the target image feature is the same, determining that the difference of the target display interface is caused by the following reasons: and the instruction coordinate included in the target instruction is inconsistent with the actual coordinate of the mobile terminal.

Further, in order to complete the cloud test of the mobile terminal, the cloud test method of the mobile terminal further includes: and performing equal-proportion shifting or line changing on the instruction coordinate according to the actual length and the actual width of the display interface of the mobile terminal, and updating the target instruction according to the shifted or line-changed instruction coordinate.

Hereinafter, the cloud testing method of the mobile terminal according to the exemplary embodiment of the present disclosure is further explained and explained with reference to fig. 5 to 7.

First, before performing a cloud test on a mobile terminal, preparation work needs to be performed on a proxy server and the mobile terminal. Referring to fig. 5, the service preparation phase may specifically include the following steps:

step S510, starting a proxy server; the proxy program of the proxy Server (Server) can start basic service at the mobile terminal, and is used for accessing and calling the proxy program of the Server terminal, downloading the mobile phone image in real time and uploading the gesture operation in real time.

Step S520, starting basic service of the mobile terminal through the proxy server; the basic service may specifically refer to fig. 6;

step S530, judging whether the basic service of the mobile terminal is started successfully; if yes, finishing the preparation work; if not, go to step S520.

Secondly, after the service preparation stage is completed, cloud testing of the mobile terminal can be achieved. Referring to fig. 7, a specific test procedure may include the following steps:

step S710, a client opens a client webpage or an application program, selects a mobile terminal needing cloud test, and accesses an agent program of an agent server;

step S720, the proxy server receives the access request sent by the client and determines the system type of the mobile terminal according to the access request;

step S730, the proxy server acquires a first current display interface from the mobile terminal and sends the first current display interface to the client;

step S740, the client generates an operation instruction according to the gesture operation of the user and sends the operation instruction to the proxy server;

and step S750, the proxy server converts the operation instruction according to the system type of the mobile terminal to obtain a target instruction, and sends the target instruction to the mobile terminal to complete the cloud test.

According to the cloud test method of the mobile terminal, the Android remote desktop access technology, the iOS remote desktop access technology and the two different mobile terminal remote desktop access technologies are unified to perform service management, API access and communication data format. According to the scheme, basic service resources of Android and iOS can be uniformly managed, a plurality of mobile phones of an Android end and an iOS end can be simultaneously managed on one server, and the effect brought by the management of the basic service resources of the Android and the iOS can be specifically shown in fig. 8.

Furthermore, on one Server, a plurality of Android terminals and a plurality of iOS terminals can be managed simultaneously, the Client terminal uses a webpage and a Client program to access the service of the Server terminal, and the remote desktop service of any one of the Android and iOS devices can be used; the remote terminal service can comprise a webpage screenshot using a remote mobile phone, a real-time picture viewing of the mobile phone and the like; and the clicking and sliding of the webpage picture area form operation data, and the operation data is finally sent to the mobile equipment service for execution.

The cloud testing method of the mobile terminal provided by the embodiment of the disclosure is mainly used for managing mobile terminal equipment by staff in a company, and is uniformly deployed in a machine room, and the staff can access and control a mobile phone in the machine room to complete corresponding development and testing work through a webpage when the staff wants to use the mobile terminal equipment for development and testing. This solution brings the following advantages: on one hand, the mobile equipment is managed in a centralized way to prevent loss; on the other hand, the use efficiency is improved, and a webpage is opened for use when the mobile equipment is used; on the other hand, a plurality of functions such as application installation and log function can be expanded, and development and test work are facilitated.

Exemplary devices

Having described the cloud test method of the mobile terminal according to the exemplary embodiment of the present disclosure, the cloud test apparatus of the mobile terminal according to the exemplary embodiment of the present disclosure will be explained and explained with reference to fig. 9.

Referring to fig. 9, the cloud testing apparatus of the mobile terminal is configured at the proxy server, and the cloud testing apparatus of the mobile terminal may request the receiving module 910, the screen acquiring module 920, the instruction converting module 930, and the cloud testing module 940. Wherein:

the request receiving module 910 may be configured to receive an access request sent by a client for a mobile terminal;

the image obtaining module 920 may be configured to, in response to the access request, obtain a first current display interface of the mobile terminal in real time based on a preset interface, and send the first current display interface to the client, so as to generate and display a second current display interface corresponding to the first current display interface on the client;

the instruction conversion module 930 may be configured to receive an operation instruction sent by the client, and convert the operation instruction according to a system type to which an operating system of the mobile terminal belongs to obtain a target instruction; wherein the operation instruction is generated by the client in response to the gesture operation aiming at the second current display interface;

the cloud testing module 940 may be configured to send the target instruction to the mobile terminal based on the preset interface, so that the mobile terminal executes the target instruction in the first current display interface to implement a cloud testing task for the mobile terminal.

In an example embodiment of the present disclosure, the cloud testing apparatus of the mobile terminal further includes:

In an example embodiment of the present disclosure, determining a cause of dissimilarity of the target display interface according to the target image feature includes:

In an example embodiment of the present disclosure, if the category of the popup is a system popup, determining that a cause of the dissimilarity of the target display interface is: a system popup is arranged on the upper layer of the target display interface, so that the target instruction cannot be executed in the current display interface;

In an example embodiment of the present disclosure, determining a cause of dissimilarity of the target display interface according to the target image feature further includes:

In an example embodiment of the present disclosure, if the comparison result of the current image feature and the target image feature is the same, it is determined that the reason causing the difference of the target display interface is: and the instruction coordinate included in the target instruction is inconsistent with the actual coordinate of the mobile terminal.

In an example embodiment of the present disclosure, the system class includes an android operating system or an iOS operating system;

In an example embodiment of the present disclosure, converting the gesture operation, the original abscissa and the original ordinate according to a data format required by the android operating system to obtain a first target instruction required by a mobile terminal having the android operating system includes:

Exemplary storage Medium

Having described the cloud test method of the mobile terminal and the cloud test apparatus of the mobile terminal according to the exemplary embodiments of the present disclosure, a storage medium according to the exemplary embodiments of the present disclosure will be described with reference to fig. 10.

Referring to fig. 10, a program product 1000 for implementing the above method according to an embodiment of the present disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium.

Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN).

Exemplary electronic device

Having described the storage medium of the exemplary embodiment of the present disclosure, next, an electronic device of the exemplary embodiment of the present disclosure is explained with reference to fig. 11.

The electronic device 1100 shown in fig. 11 is only an example and should not bring any limitations to the function and scope of use of the embodiments of the present disclosure.

As shown in fig. 11, electronic device 1100 is embodied in the form of a general purpose computing device. The components of the electronic device 1100 may include, but are not limited to: the at least one processing unit 1110, the at least one memory unit 1120, a bus 1130 connecting different system components (including the memory unit 1120 and the processing unit 1110), and a display unit 1140.

Wherein the storage unit 1120 stores program code that can be executed by the processing unit 1110 to cause the processing unit 1110 to perform the steps according to various exemplary embodiments of the present disclosure described in the above section "exemplary method" of the present specification. For example, the processing unit 1110 may perform steps S110-S140 as shown in fig. 1.

The memory unit 1120 may include a volatile memory unit such as a random access memory unit (RAM)11201 and/or a cache memory unit 11202, and may further include a read only memory unit (ROM) 11203.

Storage unit 1120 may also include a program/utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 1130 may include a data bus, an address bus, and a control bus.

The electronic device 1100 may also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, bluetooth device, etc.) via an input/output (I/O) interface 1150. Also, the electronic device 1100 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 1160. As shown, the network adapter 1160 communicates with the other modules of the electronic device 1100 over the bus 1130. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

It should be noted that although in the above detailed description several modules or sub-modules of the pop-up window processing apparatus are mentioned, such division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

It should be noted that although in the above detailed description several units/modules or sub-units/modules of the apparatus are mentioned, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more of the units/modules described above may be embodied in one unit/module, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one unit/module described above may be further divided into embodiments by a plurality of units/modules.

Further, while the operations of the disclosed methods are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.

While the spirit and principles of the present disclosure have been described with reference to several particular embodiments, it is to be understood that the present disclosure is not limited to the particular embodiments disclosed, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A cloud testing method of a mobile terminal is configured on a proxy server, and comprises the following steps:

receiving an access request to a mobile terminal sent by a client;

2. The cloud test method of a mobile terminal according to claim 1, wherein the cloud test method of a mobile terminal further comprises:

3. The cloud testing method of the mobile terminal according to claim 2, wherein determining the cause of the dissimilarity of the target display interface according to the target image features comprises:

4. The cloud testing method of the mobile terminal according to claim 3, wherein if the category of the popup is a system popup, determining that a cause of the discrepancy of the target display interface is: a system popup is arranged on the upper layer of the target display interface, so that the target instruction cannot be executed in the current display interface;

5. The cloud test method of the mobile terminal according to claim 4, wherein the cloud test method of the mobile terminal further comprises:

6. The cloud test method of the mobile terminal according to claim 4, wherein the cloud test method of the mobile terminal further comprises:

7. A cloud testing device of a mobile terminal is configured at a proxy server, and comprises:

8. A cloud test system of a mobile terminal, comprising:

a proxy server connected to the client network, for implementing the cloud testing method of the mobile terminal according to any one of claims 1 to 6;

9. A computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the cloud testing method of the mobile terminal of any one of claims 1-6.

10. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the cloud testing method of the mobile terminal of any of claims 1-6 via execution of the executable instructions.