CN113392007B - Compatibility testing method and device - Google Patents
Compatibility testing method and device Download PDFInfo
- Publication number
- CN113392007B CN113392007B CN202110674883.4A CN202110674883A CN113392007B CN 113392007 B CN113392007 B CN 113392007B CN 202110674883 A CN202110674883 A CN 202110674883A CN 113392007 B CN113392007 B CN 113392007B
- Authority
- CN
- China
- Prior art keywords
- slice
- recording
- interface elements
- path
- image segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the invention provides a method and a device for testing compatibility, wherein the method comprises the following steps: image segmentation is carried out on the global image corresponding to the user interface so as to obtain a plurality of interface elements; constructing slices in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slices and recording actions; performing the recording action aiming at interface elements in the user interface, and recording a slice conversion path corresponding to the recording action; assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path; and playing back the recording action to obtain a test result. The risk of directly carrying out contrast matching on the global image can be reduced through image segmentation, and a new path can be formed by assembling the self-organization according to the existing path, so that the problem that a multi-path guided scene cannot be intelligently selected and input is solved.
Description
Technical Field
The present invention relates to the technical field of software applications, and in particular, to a method and an apparatus for testing compatibility.
Background
In order to improve the testing efficiency and reduce the repetitive labor, a UI (User Interface) test of mobile phone software is often performed by recording input actions and playing back the recorded input actions. In the process of testing operation of the user, the operation details of the user are recorded according to time sequence, the user does not need to manually operate when testing operation is performed next time, the recorded operation is only required to be played for one time according to the recorded time sequence, and if the result after the playback operation is the same as the recorded result, the test is successful; if the result after the playback operation is different from the recorded result, the test fails, so that the purpose of one-time recording and multiple use can be achieved.
However, if the above test method is simply adopted, after one test case is recorded, the test case can only be hard to adapt to the mobile phone with the recording environment, and once the software has differences in the adapted UI layout on the mobile phones with different resolutions, the hard recording action is hard to adapt to the new environment.
Disclosure of Invention
In view of the above manner of performing UI test directly by recording an input action and playing back the recorded input action, if the adapted UI layout on the mobile phone is different, the recorded action is difficult to adapt to the new UI layout environment problem, the embodiments of the present invention are provided to provide a method for testing compatibility and a corresponding apparatus for testing compatibility, which overcome the above problem or at least partially solve the above problem.
The embodiment of the invention discloses a compatibility testing method, which comprises the following steps:
optionally, image segmentation is performed on the global image corresponding to the user interface to obtain a plurality of interface elements;
constructing slices in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slices and recording actions;
performing the recording action aiming at interface elements in the user interface, and recording a slice conversion path corresponding to the recording action;
assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and playing back the recording action to obtain a test result.
Optionally, the recording the slice conversion path corresponding to the recording action includes:
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action;
generating a slice transition path from the first slice to the second slice;
and recording a slice conversion path corresponding to the recording action.
Optionally, the recording action includes a triggering position of recording and switching of a scene in the user interface, and the determining, according to the recording action, a first slice including an interface element before switching, and a second slice including an interface element after switching includes:
Image segmentation is carried out on the image of the user interface to obtain a plurality of interface elements;
determining interface elements before switching and interface elements after switching according to the triggering position and the recorded switching of the scene in the user interface;
a first slice containing interface elements before switching and a second slice containing interface elements after switching are determined.
Optionally, the slice transition path exhibits a chain-like structure.
Optionally, the assembling to generate an unrecorded slice conversion path according to the mapping relation table and the slice conversion path includes:
determining a common path between the slices according to a slice transition path of the chain structure;
and according to the mapping relation table and the public path, assembling and generating branch paths among the slices to be used as unrecorded slice conversion paths.
Optionally, in the mapping table, one slice corresponds to at least one recording action, and one recording action corresponds to at least one slice.
Optionally, the image segmentation is performed on the global image corresponding to the user interface to obtain a plurality of interface elements, including:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm so as to obtain a plurality of interface elements.
Optionally, after the step of playing back the recording action to obtain a test result, the method further includes:
judging whether the preset image segmentation algorithm meets the requirement according to the test result;
and if the preset image segmentation algorithm does not meet the requirement, adjusting parameters of the preset image segmentation algorithm according to the test result, and returning to the step of executing the image segmentation of the global image corresponding to the user interface by adopting the preset image segmentation algorithm so as to obtain a plurality of interface elements.
Optionally, the interface elements contained in any two of the slices are present or absent from the intersection, and the interface elements contained in any two of the slices are not identical.
The embodiment of the invention also discloses a device for testing compatibility, which comprises:
the image segmentation module is used for carrying out image segmentation on the global image corresponding to the user interface so as to obtain a plurality of interface elements;
the mapping relation table establishing module is used for establishing slices in the sandbox environment according to the interface elements and establishing a mapping relation table between the slices and recording actions;
the recording module is used for carrying out the recording action aiming at the interface element in the user interface and recording a slice conversion path corresponding to the recording action;
The path assembly module is used for assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and the playback module is used for playing back the recording action to obtain a test result.
The embodiment of the invention also discloses an electronic device, which comprises:
a processor and a storage medium storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions when the electronic device is running to perform a method according to any one of the embodiments of the invention.
The embodiment of the invention also discloses a computer readable storage medium, wherein the storage medium is stored with a computer program, and the computer program is executed by a processor to execute the method according to any one of the embodiments of the invention.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, the global image is converted into the relatively targeted or observable or operable abstract element through image segmentation, so that the identification error caused by pure global image matching is effectively reduced. The abstract elements are utilized to construct slices in the sandbox environment, macroscopic operations in a user interface are converted into a logical operation sequence in the sandbox environment, so that an upper layer can be guided to conduct more refined intelligent selection, the stability and reusability of a recording/playback test mode are remarkably improved, and although the test environment is changed, stability of a test can be ensured as long as the elements abstracted through image segmentation are not changed. Moreover, the sandbox mode can greatly improve maintainability of the compatibility test system, and when a test environment is changed generally, for example, only a part of segmentation elements are changed, the sandbox environment can be ensured only by locally modifying abstract elements of the sandbox environment, and the test operation based on the sandbox environment is unchanged.
Drawings
In order to more clearly illustrate the technical solutions of the present invention, the drawings that are needed in the description of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
Fig. 1 is a schematic flow chart of recording and playback according to an embodiment of the present invention;
FIG. 2 is a flow chart of steps of a method for compatibility testing provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of a global image provided by an embodiment of the present invention;
fig. 4 is a schematic diagram of a gray scale map corresponding to a global image according to an embodiment of the present invention;
fig. 5 is a schematic diagram of an effect of noise reduction processing on a gray scale according to an embodiment of the present invention;
FIG. 6A is a schematic diagram of an interface element provided by an embodiment of the present invention;
FIG. 6B is a schematic diagram of an interface element provided by an embodiment of the present invention;
FIG. 6C is a schematic diagram of an interface element provided by an embodiment of the present invention;
FIG. 7 is a schematic illustration of a constructed slice provided by an embodiment of the present invention;
FIG. 8 is a schematic diagram of a slicing and recording relationship provided by an embodiment of the present invention;
FIG. 9 is a schematic diagram of a mapping table according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a slice transition path provided by an embodiment of the present invention;
FIG. 11 is a schematic diagram of another slice transition path provided by an embodiment of the present invention;
FIG. 12 is a schematic diagram of another slice transition path provided by an embodiment of the present invention;
FIG. 13 is a flow chart of steps of a method for compatibility testing provided by an embodiment of the present invention;
FIG. 14 is a schematic diagram of a recording and playback system provided by an embodiment of the present invention;
FIG. 15 is a block diagram of a device for compatibility testing according to an embodiment of the present invention;
FIG. 16 is a block diagram of an electronic device of the present invention;
fig. 17 is a block diagram of a computer-readable storage medium of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It will be apparent that the described embodiments are some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In order to solve the problem that the recorded actions are difficult to adapt to a new UI layout environment if the UI layout on the mobile phone is different in the manner of directly performing UI test by recording the input actions and playing back the recorded input actions. In the prior art, there have been two improved solutions: one is to use a UI path lookup API (Application Programming Interface ) provided by the handset platform itself to accurately find the location of the UI. However, for the self-drawing control without the UI path, the position of the self-drawing control cannot be found accurately, so that the method is more dependent on the API provided by the development platform, and if the object to be operated cannot be acquired accurately by the API, the method cannot be adopted for testing.
And the other is to record the neighborhood image of the input drop point in the recording process by adopting an image recognition method, and search the neighborhood image in a full screen mode according to the recorded image during playback so as to find the position of the UI. However, this method of image recognition has two limitations: the method is that a local area image is searched in a full-screen range, and in the long-term use process, the situation that two areas which are more than or equal to each other are searched in the full-screen range and match with the neighborhood image is found, which is caused by the limitation of image identification; another is that when a different situation occurs in the second time of the response result for the operation path of the same UI, it may be difficult to correctly select an appropriate subsequent operation at the time of playback and continue to play back the test case. As shown in fig. 1, two operation paths exist for clicking the control a, namely, the occurrence of the control B and the control C, and the recording process is as follows: a appearing, click A, B appearing, click B. And the process at playback is: a occurs→clicks a→c occurs, and since B occurs after clicking a during recording instead of C, C occurs at playback, which makes it difficult to determine the operations that need to be performed subsequently.
According to the scheme for solving the problems, the global image corresponding to the original test environment is abstracted into the relatively targeted or observable or operable abstract elements through the sandbox mode, so that the risk of directly comparing and matching the global image is reduced, a new path can be formed by assembling the self-organization according to the existing path, and the problem that a multi-path guided scene cannot be intelligently selected and input is solved. Specifically, in the embodiment of the invention, a plurality of interface elements are obtained by performing image segmentation on the global image corresponding to the user interface; constructing a slice in a sandbox environment according to a plurality of interface elements, and establishing a mapping relation table between the slice and the recording action; recording an interface element in the user interface, and recording a slice conversion path corresponding to the recording action; assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path; and playing back the recording action to obtain a test result. The global image can be converted into relatively targeted or observable or operable abstract elements through image segmentation, so that recognition errors caused by pure global image matching are effectively reduced. The abstract elements are utilized to construct slices in the sandbox environment, macroscopic operations in a user interface are converted into a logical operation sequence in the sandbox environment, so that an upper layer can be guided to conduct more refined intelligent selection, the stability and reusability of a recording/playback test mode are remarkably improved, and although the test environment is changed, stability of a test can be ensured as long as the elements abstracted through image segmentation are not changed. Moreover, the sandbox mode can greatly improve maintainability of the compatibility test system, and when a test environment is changed generally, for example, only a part of segmentation elements are changed, the sandbox environment can be ensured only by locally modifying abstract elements of the sandbox environment, and the test operation based on the sandbox environment is unchanged.
Referring to fig. 2, a flowchart illustrating steps of an embodiment of a method for testing compatibility according to an embodiment of the present invention may specifically include the following steps:
step S201, image segmentation is carried out on the global image corresponding to the user interface so as to obtain a plurality of interface elements;
the embodiment of the invention can be applied to a mobile terminal, the mobile terminal can comprise various mobile devices, such as a mobile Phone, a tablet computer, a game machine, a PDA and the like, and a mobile terminal operating system can comprise Android (Android), IOS, windows Phone, windows and the like, and can generally support the running of various application programs. By running an application program on the mobile terminal and rendering a User Interface (UI) on a display screen of the mobile terminal, icons corresponding to various UI controls can be contained on the User Interface, and the icons can receive User operations to trigger starting of functions corresponding to the UI controls.
In the embodiment of the invention, when the compatibility test is required to be carried out on the UI layout of the application program, the global image corresponding to the user interface of the application program can be obtained so as to carry out image segmentation on the global image corresponding to the user interface to obtain a plurality of interface elements. Specifically, a global snapshot can be performed on a user interface of an application program to obtain a global image corresponding to the user interface, and further, the global image is subjected to image segmentation to segment the image into a plurality of interface elements.
Image segmentation is a technique and process of dividing an image into a number of specific regions with unique properties and extracting objects of interest. The existing image segmentation algorithms are mainly classified into the following categories: threshold-based segmentation algorithms, region-based segmentation algorithms, edge-based segmentation algorithms, and segmentation algorithms based on specific theory, etc. In a specific implementation, an image segmentation algorithm can be selected according to actual needs to segment images, for example, a watershed algorithm can be adopted to segment images, the watershed algorithm is a segmentation algorithm based on regions, the basic idea is that the images are regarded as topological and geodesic features, the gray value of each pixel in the images represents the altitude of the point, each local minimum and the influence region thereof are called a water collecting basin, the boundary of the water collecting basin forms a watershed, the concept and formation of the watershed can be described by simulating a dipping process, a small hole is pierced on the surface of each local minimum, then the whole model is slowly immersed in human water, and as the dipping deepens, the influence region of each local minimum is gradually expanded outwards, and a dam is constructed at the junction of the two water collecting basins, namely the watershed is formed. In addition, some other image segmentation algorithms may be used to segment the global image, which is not limited in this embodiment of the present invention.
It should be noted that, the interface elements may refer to constituent elements of the user interface, for example, application icons such as a phone, a short message, a camera, a setting and a gallery are included in the mobile phone interface, and after the global image of the mobile phone interface is subjected to image segmentation, the application icons are the interface elements that need to be segmented. As an example, as shown in fig. 3, a schematic diagram of a global image is shown, the global image in fig. 3 is subjected to image segmentation by a watershed algorithm, first, the global image is converted into a gray image, as shown in fig. 4, the gray image in fig. 4 has some noise interference, the gray image can be subjected to image preprocessing, the noise in the image is reduced, so that the processed image is clearer, as shown in fig. 5, the image obtained after the noise reduction processing is performed on the gray image shown in fig. 4, and compared with fig. 4, the interference noise in the image in fig. 5 is reduced, so that the accuracy of the subsequent image segmentation can be improved, as shown in fig. 6A-6C, and the schematic diagram of an interface element obtained by performing the image segmentation on the image shown in fig. 5 is shown.
Step S202, constructing a slice in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slice and the recording action;
Specifically, all the interface elements obtained by segmentation can be utilized to construct slices in the sandbox environment, and each slice contains at least one interface element. The interface elements contained in any two slices are in or out of intersection, and the interface elements contained in any two slices are not identical. In a specific implementation, elements which can change after being switched to another interface through one operation on the same scene interface are divided into one slice, for example, the interface is provided with a sun and a car, after the car is clicked, the car disappears and the sun becomes a moon, elements corresponding to the sun and the car form one slice, and elements corresponding to the moon form one slice.
As an example, as shown in fig. 7, there are four interface elements, namely, element a, element B, element C and element D, and 3 slices are constructed by using the four interface elements, wherein slice 1 contains element a and element B, slice 2 contains element a and element C, and slice 3 contains element B and element D.
In a specific implementation, the switching of the scene in the user interface can be equivalent to the switching between slices, and the result of the operation on the interface element is the switching between slices. As shown in fig. 8, all slices together form a sandbox environment, where switching between slices is driven by a recording action, and what recording action is selected to perform depends on the interface element components contained in the slice prior to performing the recording action.
In the embodiment of the invention, the mapping table between the slice and the recording action can be established according to the interface elements contained in the slice. Recording operation performed by the recording action pointer on the interface element in the user interface, and recording a switching process of a scene in the user interface in real time when the user input operation is performed on the interface element. Because more than one operation method exists in one slice in the sandbox environment, one slice can correspond to at least one recording action in the mapping relation table, one recording action can correspond to at least one slice, and different sandbox scenes can adopt the sandbox method. As an example, as shown in a schematic diagram of a mapping table in fig. 9, slice 1 corresponds to two recording actions, recording action 1 and recording action 2, respectively, and recording action 3 corresponds to two slices, recording action 2 and recording action 3, respectively.
Step 203, performing the recording action for the interface element in the user interface, and recording a slice conversion path corresponding to the recording action;
specifically, for each recording action in the mapping relation table, a target slice corresponding to the recording action may be determined, the recording action is performed on each interface element included in the target slice, and then a slice conversion path corresponding to the recording action is recorded.
As an example, for the mapping table as shown in fig. 9, when recording operation 1 is performed under the condition of slice 1, there may be generated two cases of switching to slice 2 and slice 3, in which case, recording operation 2 is operated for slice 2 and then switching to slice 3, recording operation 3 is operated for slice 3 and then switching to slice 4, and the recorded slice switching paths are shown in fig. 10.
Step S204, assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
in order to solve the problem that the subsequent operation to be performed is difficult to determine when the operation is performed under multiple results, in the embodiment of the invention, the unrecorded slice conversion path is assembled and generated according to the mapping relation table and the slice conversion path so as to guide the scene to be tested guided by the multiple-effect path.
In a specific implementation, all possible paths may be retrieved based on a common path in the slice switch paths, and then the unrecorded slice switch paths may be generated based on the possible path assembly.
Step S205, playback the recording action to obtain a test result.
In the embodiment of the invention, all recorded slice conversion paths jointly form the content in all sandboxes, all recording actions are converted into the conversion of slices in the sandboxes, and the recording actions are played back in the sandboxes to obtain the test result.
Specifically, a test case can be generated according to the mapping relation table and the slice conversion path, and when playback is performed, recording actions are performed according to the test case to obtain a test result, so that whether the compatibility of the application program meets the requirement can be judged according to the test result, and when the compatibility of the application program does not meet the requirement, corresponding adjustment is performed in time.
In a preferred embodiment of the present invention, the step S203 may specifically include the following sub-steps:
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action; generating a slice transition path from the first slice to the second slice; and recording a slice conversion path corresponding to the recording action.
In the embodiment of the invention, a first slice containing an interface element before switching and a second slice containing an interface element after switching can be determined according to a recording action, specifically, a trigger position and a scene conversion in a user interface are recorded in the recording action, the interface element before switching can be determined according to the trigger position, and the interface element after switching can be determined according to the scene after conversion in the user interface, so that the first slice containing the interface element before switching and the second slice containing the interface element after switching can be further determined, then a slice conversion path from the first slice to the second slice is generated, and the slice conversion path corresponding to the recording action is recorded.
In a preferred embodiment of the present invention, the recording action includes a triggering position of recording and a switching of a scene in the user interface, and the determining, according to the recording action, a first slice including an interface element before switching, and a second slice including an interface element after switching includes:
image segmentation is carried out on the image of the user interface to obtain a plurality of interface elements; determining interface elements before switching and interface elements after switching according to the recorded trigger position and the switching of scenes in the user interface; a first slice containing interface elements before switching and a second slice containing interface elements after switching are determined.
Specifically, after the recording action is performed, an image of the user interface at the beginning of recording may be subjected to image segmentation to obtain a plurality of interface elements, and then the interface elements before switching are determined according to the trigger position. And determining the scene in the user interface after the switching according to the switching of the scene in the user interface, and performing image segmentation on the image of the user interface after the switching, so as to determine the interface element after the switching. Further, a first slice containing the interface element before the switching and a second slice containing the interface element after the switching may be determined.
In an embodiment of the invention, the slice transition path is represented as a chain-like structure.
Specifically, the recording actions are arranged in a chain-like structure, and the corresponding slice transformations are also arranged in a chain-like structure, i.e. the slice transformation path is represented as a chain-like structure.
In a preferred embodiment of the present invention, the step S204 may specifically include the following sub-steps:
determining a common path between the slices according to a slice transition path of the chain structure; and according to the mapping relation table and the public path, assembling and generating branch paths among the slices to be used as unrecorded slice conversion paths.
Specifically, since the slice conversion paths are represented as a chain structure, after recording a plurality of slice conversion paths, a common path between slices is determined according to the slice conversion paths of the chain structure, then triggered from the common path, possible paths are retrieved according to the mapping relation table, and branch paths between the slices are assembled and generated as unrecorded slice conversion paths.
As an example, the slice transition path recorded in fig. 10 may be converted into a path as shown in fig. 11, both paths may pass through slice 1, may be searched according to the mapping relation table from slice 1, searching may find that in the path of the first line, recording action 2 is performed on slice 2 and then switching to slice 3, whereas in the mapping relation, recording action for slice 3 includes recording action 3 in addition to recording action 2, and switching to slice 4 after recording action 3 is performed on slice 3 as known by the path of the second line recorded in fig. 10, therefore, a branch path from slice 3 in the path of the first line to slice 4 in the path of the second line may be supplemented as an unrecorded slice transition path, as shown in fig. 12, a transition from slice 2 to slice 4 may be realized as a supplemented slice transition path from a dotted arrow.
In a preferred embodiment of the present invention, the step S201 may specifically include the following sub-steps:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm so as to obtain a plurality of interface elements.
The preset image segmentation algorithm may be a preset algorithm for image segmentation, for example, a watershed algorithm. The preset image segmentation algorithm has corresponding parameters, for different settings of the parameters, the granularity of elements obtained by image segmentation is determined, in a specific implementation, the parameters of the preset image segmentation algorithm are fixed after the parameters of the preset image segmentation algorithm are adjusted to a proper value by training the preset image segmentation algorithm, and the preset image segmentation algorithm with the fixed parameters is adopted to carry out image segmentation on the global image corresponding to the user interface so as to obtain a plurality of interface elements.
Referring to fig. 13, a flowchart illustrating steps of another method embodiment of compatibility testing provided by the embodiment of the present invention may specifically include the following steps:
step S1301, performing image segmentation on the global image corresponding to the user interface by using a preset image segmentation algorithm to obtain a plurality of interface elements;
For example, the preset image segmentation algorithm may be a watershed algorithm, and the global image corresponding to the user interface may be subjected to image segmentation by the watershed algorithm to obtain a plurality of interface elements.
Step S1302, constructing slices in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slices and recording actions;
further, the slice in the sandbox environment may be constructed according to all the interface elements, and the mapping relation table between the slice and the recording action is basically similar to the step S202 described above in the embodiment of the present invention, and will not be described herein.
Step S1303, performing the recording action with respect to the interface element in the user interface, and recording a slice conversion path corresponding to the recording action;
step S1303 in the embodiment of the present invention is substantially similar to step S203 described above, and will not be described herein.
Step S1304, assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
step S1304 in the embodiment of the present invention is substantially similar to step S204 described above, and will not be described herein.
Step S1305, playing back the recording action to obtain a test result;
Step S1305 in the embodiment of the present invention is substantially similar to step S205 described above, and will not be described herein.
Step S1306, judging whether the preset image segmentation algorithm meets the requirement according to the test result;
in the embodiment of the invention, all recorded contents in all sandboxed environments are formed together through all recorded slice conversion paths, all recorded actions are converted into the conversion of slices in the sandboxed environments, however, the granularity of interface elements in the slices is automatically generated by a preset image segmentation algorithm, so that the segmentation of the interface elements directly influences the construction quality of the sandboxed environments. In order to improve the construction quality of the sandbox environment, in the embodiment of the invention, parameters of a preset image segmentation algorithm corresponding to a test result in playback can be adjusted.
Specifically, by regarding the recording playback itself as a black box, when the parameters of the preset image segmentation algorithm are adjusted, the parameters of the preset image segmentation algorithm are adjusted with the playback quality for the recording action as feedback.
In a specific implementation, when some operations exist in the test result that the path cannot be determined, and the corresponding recording operation cannot be performed on the slice, the preset image segmentation algorithm is considered to be unsatisfied, and parameters of the preset image segmentation algorithm need to be adjusted. And when the operation that the path cannot be determined does not exist in the test result, the preset image segmentation algorithm is considered to meet the requirement.
Step S1307, if the preset image segmentation algorithm does not meet the requirement, adjusting parameters of the preset image segmentation algorithm according to the test result, and returning to the step of executing the image segmentation on the global image corresponding to the user interface by adopting the preset image segmentation algorithm to obtain a plurality of interface elements.
In the embodiment of the present invention, if the preset image segmentation algorithm does not meet the requirement, the parameters of the preset image segmentation algorithm may be adjusted according to the test result, and the execution step S1201 is returned until the preset image segmentation algorithm is determined to meet the requirement.
After judging that the preset image segmentation algorithm meets the requirement, as shown in fig. 14, after the system is stable, the parameters of the preset image segmentation algorithm can be adjusted to a proper size, after the adjustment is finished, the parameters are fixed, and the test work of recording and playback is completed by using the sandbox environment.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 15, a block diagram of a device for testing compatibility according to an embodiment of the present invention is shown, which may specifically include the following modules:
an image segmentation module 1501, configured to perform image segmentation on a global image corresponding to a user interface to obtain a plurality of interface elements;
a mapping relation table establishing module 1502, configured to construct a slice in a sandbox environment according to the plurality of interface elements, and establish a mapping relation table between the slice and a recording action;
a recording module 1503, configured to perform the recording action with respect to an interface element in the user interface, and record a slice conversion path corresponding to the recording action;
a path assembling module 1504, configured to assemble and generate an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and a playback module 1505 for playing back the recording action to obtain a test result.
In a preferred embodiment of the present invention, the recording module 1503 includes:
a slice determination submodule for determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action;
a path generation sub-module for generating a slice transition path from the first slice to the second slice;
And the path recording sub-module is used for recording the slice conversion path corresponding to the recording action.
In a preferred embodiment of the present invention, the recording action includes a trigger position of recording and a switching of scenes in the user interface, and the slice determination submodule includes:
the image segmentation unit is used for carrying out image segmentation on the image of the user interface to obtain a plurality of interface elements;
the interface element determining unit is used for determining interface elements before switching and interface elements after switching according to the triggering position and the recorded switching of the scene in the user interface;
and the slice determining unit is used for determining a first slice containing the interface element before switching and a second slice containing the interface element after switching.
In a preferred embodiment of the invention, the slice transition path is represented as a chain-like structure.
In a preferred embodiment of the present invention, the path assembly module 1504 includes:
a common path determination submodule for determining a common path between the slices according to a slice transition path of the chain structure;
and the path assembly sub-module is used for assembling and generating branch paths among the slices according to the mapping relation table and the public paths to be used as unrecorded slice conversion paths.
In a preferred embodiment of the present invention, in the mapping table, one of the slices corresponds to at least one of the recording actions, and one of the recording actions corresponds to at least one of the slices.
In a preferred embodiment of the present invention, the image segmentation module 1501 includes:
the image segmentation sub-module is used for carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm so as to obtain a plurality of interface elements.
In a preferred embodiment of the present invention, further comprising:
the judging module is used for judging whether the preset image segmentation algorithm meets the requirement according to the test result;
and the parameter adjustment module is used for adjusting parameters of the preset image segmentation algorithm according to the test result if the preset image segmentation algorithm does not meet the requirement, and returning to the step of executing the image segmentation of the global image corresponding to the user interface by adopting the preset image segmentation algorithm so as to obtain a plurality of interface elements.
In a preferred embodiment of the invention, the interface elements contained in any two of said slices are present or absent and the interface elements contained in any two of said slices are not identical.
For the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments for relevant points.
The embodiment of the invention also provides an electronic device, as shown in fig. 16, including:
a processor 1601 and a storage medium 1602, the storage medium 1602 storing machine-readable instructions executable by the processor 1601, the processor 1601 executing the machine-readable instructions to perform a method as described in any of the embodiments of the present invention when the electronic device is running. The specific implementation manner and the technical effect are similar, and are not repeated here.
The embodiment of the present invention further provides a computer readable storage medium, as shown in fig. 17, where a computer program 1701 is stored, and the computer program 1701, when executed by a processor, performs a method according to any one of the embodiments of the present invention. The specific implementation manner and the technical effect are similar, and are not repeated here.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has described in detail a method and an apparatus for compatibility testing according to the present invention, and specific examples have been provided herein to illustrate the principles and embodiments of the present invention, the above examples being provided only to assist in understanding the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (9)
1. A method of compatibility testing, comprising:
image segmentation is carried out on the global image corresponding to the user interface so as to obtain a plurality of interface elements;
constructing slices in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slices and recording actions; the slice comprises at least one interface element, any two interface elements contained in the slice have intersection or do not have intersection, and the interface elements contained in any two slices are not identical;
performing the recording action aiming at interface elements in the user interface, and recording a slice conversion path corresponding to the recording action;
according to the mapping relation table and the slice conversion path, assembling to generate an unrecorded slice conversion path so as to guide a scene to be tested guided by a multi-effect path;
playing back the recording action to obtain a test result;
the recording action comprises a triggering position of recording and switching of scenes in the user interface, and the recording of a slice conversion path corresponding to the recording action comprises the following steps:
image segmentation is carried out on the image of the user interface to obtain a plurality of interface elements;
Determining interface elements before switching and interface elements after switching according to the triggering position and the recorded switching of the scene in the user interface;
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching;
generating a slice transition path from the first slice to the second slice;
and recording a slice conversion path corresponding to the recording action.
2. The method of claim 1, wherein the slice transition path exhibits a chain-like structure.
3. The method of claim 2, wherein assembling to generate an unrecorded slice transition path according to the mapping table and the slice transition path comprises:
determining a common path between the slices according to a slice transition path of the chain structure;
and according to the mapping relation table and the public path, assembling and generating branch paths among the slices to be used as unrecorded slice conversion paths.
4. The method of claim 1 wherein one of said slices corresponds to at least one of said recording actions and one of said recording actions corresponds to at least one of said slices in said mapping table.
5. The method of claim 1, wherein the image segmentation of the global image corresponding to the user interface to obtain a plurality of interface elements comprises:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm so as to obtain a plurality of interface elements.
6. The method of claim 5, further comprising, after the step of playing back the recording action to obtain a test result:
judging whether the preset image segmentation algorithm meets the requirement according to the test result;
and if the preset image segmentation algorithm does not meet the requirement, adjusting parameters of the preset image segmentation algorithm according to the test result, and returning to the step of executing the image segmentation of the global image corresponding to the user interface by adopting the preset image segmentation algorithm so as to obtain a plurality of interface elements.
7. An apparatus for compatibility testing, comprising:
the image segmentation module is used for carrying out image segmentation on the global image corresponding to the user interface so as to obtain a plurality of interface elements;
the mapping relation table establishing module is used for establishing slices in the sandbox environment according to the interface elements and establishing a mapping relation table between the slices and recording actions; the slice comprises at least one interface element, any two interface elements contained in the slice have intersection or do not have intersection, and the interface elements contained in any two slices are not identical;
The recording module is used for carrying out the recording action aiming at the interface element in the user interface and recording a slice conversion path corresponding to the recording action;
the path assembly module is used for assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path so as to guide a scene to be tested guided by the multi-result path;
the playback module is used for playing back the recording action to obtain a test result;
the recording action comprises a triggering position of recording and switching of scenes in the user interface, and the recording module is further used for carrying out image segmentation on images of the user interface to obtain a plurality of interface elements; determining interface elements before switching and interface elements after switching according to the triggering position and the recorded switching of the scene in the user interface; determining a first slice containing interface elements before switching and a second slice containing interface elements after switching; generating a slice transition path from the first slice to the second slice; and recording a slice conversion path corresponding to the recording action.
8. An electronic device, comprising:
a processor and a storage medium storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions when the electronic device is running to perform the method of any one of claims 1-6.
9. A computer readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a processor, performs the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674883.4A CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674883.4A CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113392007A CN113392007A (en) | 2021-09-14 |
| CN113392007B true CN113392007B (en) | 2023-09-08 |
Family
ID=77621716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110674883.4A Active CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392007B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365779A (en) * | 2013-07-09 | 2013-10-23 | 广东欧珀移动通信有限公司 | Software test method and system |
| WO2017049649A1 (en) * | 2015-09-26 | 2017-03-30 | Intel Corporation | Technologies for automated application exploratory testing |
| CN108052261A (en) * | 2017-12-07 | 2018-05-18 | 广东欧珀移动通信有限公司 | Record screen method, apparatus and terminal |
| CN109189665A (en) * | 2018-08-01 | 2019-01-11 | 阿里巴巴集团控股有限公司 | Data recording, data readback and the method and device tested automatically |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| CN109857674A (en) * | 2019-02-27 | 2019-06-07 | 上海优扬新媒信息技术有限公司 | A kind of recording and playback test method and relevant apparatus |
| CN109871324A (en) * | 2019-01-29 | 2019-06-11 | 广州云测信息技术有限公司 | A kind of operating method and device of control |
| CN109885481A (en) * | 2019-01-14 | 2019-06-14 | 珠海金山网络游戏科技有限公司 | A kind of the recording and playback test method and device of the game NGUI based on Unity technology |
| CN110275834A (en) * | 2019-06-25 | 2019-09-24 | 中国工商银行股份有限公司 | User interface automatization test system and method |
| CN111258893A (en) * | 2020-01-13 | 2020-06-09 | 中信银行股份有限公司 | Mobile terminal application automatic testing device for randomly assembling transaction path |
| US10853218B1 (en) * | 2018-12-17 | 2020-12-01 | Shutterstock, Inc. | Recording and playback of a web browser session |
| CN112579431A (en) * | 2019-09-30 | 2021-03-30 | 南京大学 | Cross-platform script recording and playback method based on image recognition |
| WO2021088724A1 (en) * | 2019-11-05 | 2021-05-14 | 深圳前海微众银行股份有限公司 | Testing method and apparatus |
| CN112817866A (en) * | 2021-02-25 | 2021-05-18 | 北京百家科技集团有限公司 | Recording playback method, device, system, computer equipment and storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8856749B2 (en) * | 2010-09-28 | 2014-10-07 | Microsoft Corporation | Multi-path brokered test automation execution |
| US8612806B2 (en) * | 2011-12-16 | 2013-12-17 | Sap Ag | Systems and methods for recording user interactions within a target application |
| US20150169430A1 (en) * | 2013-12-13 | 2015-06-18 | International Business Machines Corporation | Selecting webpage test paths |
| US9865299B2 (en) * | 2014-11-28 | 2018-01-09 | Sony Corporation | Information processing device, information recording medium, information processing method, and program |
| US10191832B2 (en) * | 2016-11-14 | 2019-01-29 | Microsoft Technology Licensing, Llc | Multi-language playback framework |
| US10394695B2 (en) * | 2017-09-25 | 2019-08-27 | Oracle International Corporation | Method and system for recording and debugging process flows |
| CN111443964B (en) * | 2019-01-17 | 2023-10-20 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer readable storage medium for updating user interface |
-
2021
- 2021-06-17 CN CN202110674883.4A patent/CN113392007B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365779A (en) * | 2013-07-09 | 2013-10-23 | 广东欧珀移动通信有限公司 | Software test method and system |
| WO2017049649A1 (en) * | 2015-09-26 | 2017-03-30 | Intel Corporation | Technologies for automated application exploratory testing |
| CN108052261A (en) * | 2017-12-07 | 2018-05-18 | 广东欧珀移动通信有限公司 | Record screen method, apparatus and terminal |
| CN109189665A (en) * | 2018-08-01 | 2019-01-11 | 阿里巴巴集团控股有限公司 | Data recording, data readback and the method and device tested automatically |
| US10853218B1 (en) * | 2018-12-17 | 2020-12-01 | Shutterstock, Inc. | Recording and playback of a web browser session |
| CN109885481A (en) * | 2019-01-14 | 2019-06-14 | 珠海金山网络游戏科技有限公司 | A kind of the recording and playback test method and device of the game NGUI based on Unity technology |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| CN109871324A (en) * | 2019-01-29 | 2019-06-11 | 广州云测信息技术有限公司 | A kind of operating method and device of control |
| CN109857674A (en) * | 2019-02-27 | 2019-06-07 | 上海优扬新媒信息技术有限公司 | A kind of recording and playback test method and relevant apparatus |
| CN110275834A (en) * | 2019-06-25 | 2019-09-24 | 中国工商银行股份有限公司 | User interface automatization test system and method |
| CN112579431A (en) * | 2019-09-30 | 2021-03-30 | 南京大学 | Cross-platform script recording and playback method based on image recognition |
| WO2021088724A1 (en) * | 2019-11-05 | 2021-05-14 | 深圳前海微众银行股份有限公司 | Testing method and apparatus |
| CN111258893A (en) * | 2020-01-13 | 2020-06-09 | 中信银行股份有限公司 | Mobile terminal application automatic testing device for randomly assembling transaction path |
| CN112817866A (en) * | 2021-02-25 | 2021-05-18 | 北京百家科技集团有限公司 | Recording playback method, device, system, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113392007A (en) | 2021-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2999619C (en) | Application engineering platform | |
| CN111881019B (en) | User interface testing method and device | |
| US11335004B2 (en) | Generating refined segmentation masks based on uncertain pixels | |
| CN106155454B (en) | Interface display method and device and electronic equipment | |
| CN110288549B (en) | Video repairing method and device and electronic equipment | |
| KR102339205B1 (en) | Virtual scene display method and device, and storage medium | |
| CN109978044B (en) | Training data generation method and device, and model training method and device | |
| CN110287877A (en) | The processing method and processing device of video object | |
| CN113268243B (en) | Memory prediction method and device, storage medium and electronic equipment | |
| CN112241361A (en) | Test case generation method and device and problem scene automatic reproduction method and device | |
| CN116452920A (en) | Image processing method and device, storage medium and electronic equipment | |
| CN115470101A (en) | Data testing method and device, computer equipment and storage medium | |
| CN105718363B (en) | Mobile phone response starting time point acquisition methods and device | |
| US20130167087A1 (en) | Mode-based graphical editing | |
| CN113392007B (en) | Compatibility testing method and device | |
| US20200082586A1 (en) | Generating and providing composition effect tutorials for creating and editing digital content | |
| CN104093069A (en) | Video playing method and player device | |
| CN116363154A (en) | Image processing method and device | |
| CN115543168A (en) | Method, device, equipment and medium for switching application | |
| CN109088977B (en) | Control method and control device for automatically installing mobile terminal application | |
| CN110704742B (en) | Feature extraction method and device | |
| CN110968513A (en) | Recording method and device of test script | |
| CN111209501B (en) | Picture display method and device, electronic equipment and storage medium | |
| CN112987997B (en) | Path finding method of user interface menu and computer equipment | |
| US20240193843A1 (en) | Animated display characteristic control for digital images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |