CN113392007A - Compatibility testing method and device - Google Patents
Compatibility testing method and device Download PDFInfo
- Publication number
- CN113392007A CN113392007A CN202110674883.4A CN202110674883A CN113392007A CN 113392007 A CN113392007 A CN 113392007A CN 202110674883 A CN202110674883 A CN 202110674883A CN 113392007 A CN113392007 A CN 113392007A
- Authority
- CN
- China
- Prior art keywords
- slice
- recording
- interface elements
- path
- image segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the invention provides a method and a device for testing compatibility, wherein the method comprises the following steps: carrying out image segmentation on a global image corresponding to a user interface to obtain a plurality of interface elements; constructing a slice in the sandbox environment according to the plurality of interface elements, and establishing a mapping relation table between the slice and the recording action; the recording action is carried out aiming at the interface elements in the user interface, and a slice conversion path corresponding to the recording action is recorded; assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path; and playing back the recording action to obtain a test result. The risk of directly comparing and matching the global image can be reduced through image segmentation, a new path can be formed by self assembly according to the existing path, and the problem that the scene guided by multiple paths cannot be intelligently selected and input is solved.
Description
Technical Field
The present invention relates to the field of software application technologies, and in particular, to a compatibility testing method and a compatibility testing apparatus.
Background
In order to improve the testing efficiency and reduce the repetitive labor, a UI (User Interface) test of mobile phone software usually performs a UI test by recording an input action and playing back the recorded input action. The method comprises the steps that operation details of a user are recorded in time sequence in the process of user test operation, the user does not need to manually operate in the next test operation, only recorded operation needs to be played once according to the recorded time sequence, and if the result after the playback operation is finished is the same as the recorded result, the test is successful; if the result after the playback operation is not the same as the recorded result, the test fails, thus achieving the purpose of one-time recording and multiple-time use.
However, if the above test method is simply adopted, after a test case is recorded, the test case can only be adapted to the mobile phone in the recording environment in a rigid manner, and once the UI layouts adapted to the mobile phones with different resolutions of the software are different, the rigid recording action is difficult to adapt to the new environment.
Disclosure of Invention
In view of the above-mentioned manner of directly performing a UI test by recording an input action and playing back a recorded input action, if the adapted UI layouts on the mobile phone are different, the recorded action is difficult to adapt to a new UI layout environment problem, embodiments of the present invention are proposed to provide a method for a compatibility test and a corresponding apparatus for a compatibility test that overcome or at least partially solve the above-mentioned problems.
The embodiment of the invention discloses a compatibility testing method, which comprises the following steps:
optionally, performing image segmentation on a global image corresponding to the user interface to obtain a plurality of interface elements;
constructing a slice in the sandbox environment according to the plurality of interface elements, and establishing a mapping relation table between the slice and the recording action;
the recording action is carried out aiming at the interface elements in the user interface, and a slice conversion path corresponding to the recording action is recorded;
assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and playing back the recording action to obtain a test result.
Optionally, the recording a slice switching path corresponding to the recording action includes:
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action;
generating a slice transition path from the first slice to the second slice;
and recording a slice conversion path corresponding to the recording action.
Optionally, the determining, according to the recording action, a first slice including an interface element before switching and a second slice including an interface element after switching includes:
carrying out image segmentation on the image of the user interface to obtain a plurality of interface elements;
determining interface elements before switching and interface elements after switching according to the triggering position and the switching of the recorded scenes in the user interface;
a first slice containing the interface elements before switching and a second slice containing the interface elements after switching are determined.
Optionally, the slice transition path exhibits a chain-like structure.
Optionally, the assembling and generating an unrecorded slice conversion path according to the mapping relationship table and the slice conversion path includes:
determining a common path between the slices according to the slice switching path of the chain structure;
and assembling and generating branch paths among the slices as unrecorded slice conversion paths according to the mapping relation table and the common path.
Optionally, in the mapping relationship table, one of the slices corresponds to at least one of the recording actions, and one of the recording actions corresponds to at least one of the slices.
Optionally, the image segmentation is performed on the global image corresponding to the user interface to obtain a plurality of interface elements, and includes:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm to obtain a plurality of interface elements.
Optionally, after the step of playing back the recording action to obtain the test result, the method further includes:
judging whether the preset image segmentation algorithm meets the requirements or not according to the test result;
and if the preset image segmentation algorithm does not meet the requirements, adjusting the parameters of the preset image segmentation algorithm according to the test result, and returning to execute the step of performing image segmentation on the global image corresponding to the user interface by adopting the preset image segmentation algorithm to obtain a plurality of interface elements.
Optionally, there is an intersection or no intersection between the interface elements contained in any two slices, and the interface elements contained in any two slices are not identical.
The embodiment of the invention also discloses a device for testing compatibility, which comprises:
the image segmentation module is used for carrying out image segmentation on a global image corresponding to the user interface to obtain a plurality of interface elements;
the mapping relation table establishing module is used for establishing a slice in the sandbox environment according to the plurality of interface elements and establishing a mapping relation table between the slice and the recording action;
the recording module is used for carrying out the recording action aiming at the interface elements in the user interface and recording the slice conversion path corresponding to the recording action;
the path assembling module is used for assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and the playback module is used for playing back the recording action to obtain a test result.
The embodiment of the invention also discloses an electronic device, which comprises:
a processor and a storage medium storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions to perform a method according to any one of the embodiments of the invention when the electronic device is operated.
The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method of any one of the embodiments of the invention is executed.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, the global image is converted into relatively targeted, observable or operable abstract elements through image segmentation, so that the identification error caused by pure global image matching is effectively reduced. The abstract elements are used for constructing the slices in the sandbox environment, macro operation in a user interface is converted into a logical operation sequence in the sandbox environment, so that the upper layer can be guided to perform more refined intelligent selection, the stability and the reusability of the recording/playback test mode are obviously improved, and the stability of the test can be guaranteed to be unchanged as long as the abstract elements are not changed through image segmentation although the test environment is changed. Moreover, the sandbox mode can greatly improve the maintainability of the compatibility test system, and when the test environment is generally changed, for example, only a part of the segmentation elements are changed, the sandbox environment can be ensured only by locally modifying the abstract elements of the sandbox environment, and the test operation based on the sandbox environment is unchanged.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flow chart of recording and playback according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of a method for compatibility testing according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a global image according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a gray scale map corresponding to a global image according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating an effect of performing noise reduction processing on a grayscale chart according to an embodiment of the present invention;
FIG. 6A is a schematic diagram of an interface element provided by an embodiment of the present invention;
FIG. 6B is a schematic diagram of an interface element according to an embodiment of the present invention;
FIG. 6C is a schematic diagram of an interface element provided by an embodiment of the present invention;
FIG. 7 is a schematic illustration of a constructed slice provided by an embodiment of the present invention;
fig. 8 is a schematic diagram of a relationship between slicing and recording actions according to an embodiment of the present invention;
FIG. 9 is a diagram illustrating a mapping table according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of a slice transition path provided by an embodiment of the invention;
FIG. 11 is a schematic diagram of another slice transition path provided by embodiments of the present invention;
FIG. 12 is a schematic diagram of another slice transition path provided by embodiments of the present invention;
FIG. 13 is a flowchart illustrating steps in a method for compatibility testing according to an embodiment of the present invention;
fig. 14 is a schematic diagram of a recording and playback system provided by an embodiment of the present invention;
fig. 15 is a block diagram of a device for testing compatibility according to an embodiment of the present invention;
FIG. 16 is a block diagram of an electronic device of the present invention;
fig. 17 is a block diagram of a computer-readable storage medium of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The method aims to solve the problem that the recorded actions are difficult to adapt to a new UI layout environment if the adapted UI layout on the mobile phone is different in the mode of directly carrying out UI test by recording the input actions and replaying the recorded input actions. In the prior art, there have been two improved solutions: one is to precisely find the position of the UI by using a UI path search API (Application Programming Interface) provided by the mobile phone platform itself. However, the position of the self-drawing control without the UI path cannot be accurately found, so that the method is relatively dependent on the API provided by the development platform, and if the object to be operated cannot be accurately obtained by the API, the method cannot be used for testing.
And the other method is to adopt an image recognition method, record a neighborhood image of the input drop point in the recording process, and search the image in a full screen mode according to the recorded image during playback so as to find the position of the UI. However, this method of image recognition has two limitations: one is that a method of searching a local area image in a full screen range is adopted, and in the long-term use process, the condition that more than or equal to two areas are searched in the full screen range to be matched with a neighborhood image can occur, which is caused by the limitation of image identification; another is that when the response result of the same UI operation path is different for the second time, it is difficult to correctly select a suitable subsequent operation and continue playing back the test case during playback. As shown in fig. 1, there are two operation paths for clicking the control a, which are the occurrence of the control B and the control C, respectively, and the recording process includes: a appears → click A → B appears → click B. And the process during playback is as follows: a appears → a is clicked → C appears, and since B appears instead of C after a is clicked in the process at the time of recording, C appears at the time of playback, making it difficult to determine the operation to be performed subsequently.
The embodiment of the invention provides a scheme for solving the problems, a global image corresponding to an original test environment is abstracted into relatively targeted, observable or operable abstract elements through a sandbox mode so as to reduce the risk of directly comparing and matching the global image, a new path can be automatically formed through assembling according to the existing path, and the problem that a multipath-oriented scene cannot be intelligently selected and input is solved. Specifically, in the embodiment of the present invention, a plurality of interface elements are obtained by performing image segmentation on a global image corresponding to a user interface; constructing a slice in a sandbox environment according to a plurality of interface elements, and establishing a mapping relation table between the slice and the recording action; recording actions are carried out on interface elements in the user interface, and slice conversion paths corresponding to the recording actions are recorded; assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path; and playing back the recording action to obtain a test result. Therefore, the global image can be converted into relatively targeted, observable or operable abstract elements through image segmentation, and the identification error caused by pure global image matching is effectively reduced. The abstract elements are used for constructing the slices in the sandbox environment, macro operation in a user interface is converted into a logical operation sequence in the sandbox environment, so that the upper layer can be guided to perform more refined intelligent selection, the stability and the reusability of the recording/playback test mode are obviously improved, and the stability of the test can be guaranteed to be unchanged as long as the abstract elements are not changed through image segmentation although the test environment is changed. Moreover, the sandbox mode can greatly improve the maintainability of the compatibility test system, and when the test environment is generally changed, for example, only a part of the segmentation elements are changed, the sandbox environment can be ensured only by locally modifying the abstract elements of the sandbox environment, and the test operation based on the sandbox environment is unchanged.
Referring to fig. 2, a flowchart illustrating steps of an embodiment of a method for testing compatibility according to an embodiment of the present invention is shown, which may specifically include the following steps:
step S201, carrying out image segmentation on a global image corresponding to a user interface to obtain a plurality of interface elements;
the embodiment of the invention can be applied to a mobile terminal, the mobile terminal can comprise various mobile devices, such as a mobile Phone, a tablet computer, a game machine, a PDA and the like, the operating system of the mobile terminal can comprise Android (Android), IOS (input operating system), Windows Phone, Windows and the like, and the operation of various application programs can be supported generally. The method comprises the steps that an application program is run on the mobile terminal, a User Interface (UI) is rendered on a display screen of the mobile terminal, icons corresponding to various UI controls can be contained on the UI, and the icons can receive the operation of a User to trigger and start functions corresponding to the UI controls.
In the embodiment of the invention, when the compatibility test needs to be performed on the UI layout of the application program, the global image corresponding to the user interface of the application program can be obtained, so that the global image corresponding to the user interface is subjected to image segmentation to obtain a plurality of interface elements. Specifically, a global image corresponding to the user interface can be obtained by performing a global snapshot on the user interface of the application program, and further, the image is segmented into a plurality of interface elements by performing image segmentation on the global image.
Image segmentation is a technique and a process for dividing an image into a plurality of specific regions with unique properties and extracting an interested target. The existing image segmentation algorithms are mainly classified into the following categories: threshold-based segmentation algorithms, region-based segmentation algorithms, edge-based segmentation algorithms, and segmentation algorithms based on specific theories, etc. In a specific implementation, an image segmentation algorithm can be selected according to actual needs for image segmentation, for example, a watershed algorithm can be used for image segmentation, the watershed algorithm is a region-based segmentation algorithm, the basic idea is that the image is regarded as a geodetic topological landform, the gray value of each pixel in the image represents the altitude of the point, each local minimum value and the influence region thereof are called as a catchment basin, the boundary of the catchment basin forms a watershed, the concept and formation of the watershed can be described by simulating the immersion process, a small hole is pierced on the surface of each local minimum value, then the whole model is slowly immersed in human water, the influence region of each local minimum value slowly expands outwards as the immersion deepens, and a dam is constructed at the junction of two catchment basins, namely, a watershed is formed. In addition, other image segmentation algorithms may also be used to perform image segmentation on the global image, which is not limited in this embodiment of the present invention.
It should be noted that the interface elements may refer to constituent elements of the user interface, for example, the mobile phone interface includes application icons such as a phone call, a short message, a camera, a setting, and a gallery, and after the image segmentation is performed on the global image of the mobile phone interface, the application icons are interface elements that need to be segmented. As an example, as shown in fig. 3, the image segmentation is performed on the global image in fig. 3 by using a watershed algorithm, and first, the global image is converted into a gray scale map, as shown in fig. 4, there is some noise interference in the gray scale map in fig. 4, the gray scale map may be subjected to image preprocessing to reduce noise in the image, so as to make the processed image clearer, as shown in fig. 5, the image obtained by performing noise reduction processing on the gray scale shown in fig. 4 is an image, as compared with fig. 4, the interference noise in the image in fig. 5 is reduced, so that the accuracy of subsequent image segmentation may be improved, as shown in fig. 6A to fig. 6C, the image is a schematic diagram of interface elements obtained by performing image segmentation on fig. 5.
Step S202, constructing slices in a sandbox environment according to the interface elements, and establishing a mapping relation table between the slices and recording actions;
specifically, the slices in the sandbox environment may be constructed by using all interface elements obtained by the segmentation, where each slice includes at least one interface element. And the interface elements contained in any two slices have intersection or do not have intersection, and the interface elements contained in any two slices are not identical. In a specific implementation, elements that change after being switched to another interface through one operation on the same scene interface may be divided into one slice, for example, the interface has a sun and a car, after the car is clicked, the car disappears, and the sun becomes a moon, then the elements corresponding to the sun and the car constitute one slice, and the elements corresponding to the moon constitute one slice.
As an example, as shown in fig. 7, there are four interface elements, which are element a, element B, element C, and element D, and 3 slices are constructed by using the four interface elements, where slice 1 includes element a and element B, slice 2 includes element a and element C, and slice 3 includes element B and element D.
In a specific implementation, switching of a scene in a user interface may be equivalent to switching between slices, and a result of an operation on an interface element is switching between slices. As shown in fig. 8, all slices collectively constitute a sandbox environment in which switching between slices is driven by a recording action, and which recording action is selected to be performed depends on the interface element composition contained in the slice prior to performing the recording action.
In the embodiment of the invention, a mapping table between the slicing action and the recording action can be established according to the interface elements contained in the slicing action. The recording action pointer records the recording operation of the interface elements in the user interface, and the switching process of the scenes in the user interface is recorded in real time when the user input operation is performed on the interface elements. In the sandbox environment, more than one operation method exists in one slice, so that in the mapping relation table, one slice can correspond to at least one recording action, one recording action can correspond to at least one slice, and different sandbox scenes can adopt the sandbox method. As an example, as shown in fig. 9, a schematic diagram of a mapping table shows that slice 1 corresponds to two recording actions, i.e., recording action 1 and recording action 2, respectively, and recording action 3 corresponds to two slices, i.e., slice 2 and slice 3, respectively.
Step S203, the recording action is carried out aiming at the interface elements in the user interface, and a slice conversion path corresponding to the recording action is recorded;
specifically, for each recording action in the mapping relationship table, a target slice corresponding to the recording action may be determined, the recording action is performed on each interface element included in the target slice, and then a slice conversion path corresponding to the recording action is recorded.
As an example, for the mapping relation table shown in fig. 9, when the recording action 1 is executed under the condition of the slice 1, two cases of switching to the slice 2 and the slice 3 may occur, in which, in the two cases, the recording action 2 is operated for the slice 2 and then the slice 3 is switched to, and the recording action 3 is operated for the slice 3 and then the slice 4 is switched to, and the recorded slice switching path is as shown in fig. 10.
Step S204, assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
in order to solve the problem that subsequent operations which need to be performed are difficult to determine when the operations are performed under multiple results, in the embodiment of the invention, the unrecorded slice conversion path is generated by assembling according to the mapping relation table and the slice conversion path so as to guide a scene to be tested guided by the multiple result path.
In a specific implementation, all possible paths may be retrieved from a common path in the slice switching paths, and then an unrecorded slice switching path may be generated according to possible path assembly.
And step S205, playing back the recording action to obtain a test result.
In the embodiment of the invention, all recorded slice conversion paths jointly form the content in all sandbox environments, all recording actions are converted into conversion of slices in the sandbox environments, and the test results are obtained by playing back the recording actions in the sandbox environments.
Specifically, a test case can be generated according to the mapping relation table and the slice conversion path, and during playback, a recording action is executed according to the test case to obtain a test result, so that whether the compatibility of the application program meets the requirement can be judged according to the test result, and when the compatibility of the application program does not meet the requirement, corresponding adjustment can be performed in time.
In a preferred embodiment of the present invention, the step S203 may specifically include the following sub-steps:
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action; generating a slice transition path from the first slice to the second slice; and recording a slice conversion path corresponding to the recording action.
In the embodiment of the present invention, a first slice including an interface element before switching and a second slice including an interface element after switching may be determined according to a recording action, and specifically, a trigger position and a transition of a scene in a user interface are recorded in the recording action, an interface element before switching may be determined according to the trigger position, and an interface element after switching may be determined according to the scene after switching in the user interface, so that the first slice including the interface element before switching and the second slice including the interface element after switching may be further determined, a slice transition path from the first slice to the second slice is then generated, and a slice transition path corresponding to the recording action is recorded.
In a preferred embodiment of the present invention, the determining, according to the recording action, a first slice including an interface element before switching and a second slice including an interface element after switching, where the recording action includes a trigger position for recording and switching of a scene in the user interface includes:
carrying out image segmentation on the image of the user interface to obtain a plurality of interface elements; determining interface elements before switching and interface elements after switching according to the recorded triggering position and the switching of scenes in the user interface; a first slice containing the interface elements before switching and a second slice containing the interface elements after switching are determined.
Specifically, after the recording action is performed, the image of the user interface at the beginning of recording may be image-divided to obtain a plurality of interface elements, and then the interface element before switching may be determined according to the trigger position. And according to the switching of the scenes in the user interface, determining the scenes in the user interface after the switching, and performing image segmentation on the images of the user interface after the switching so as to determine the interface elements after the switching. Further, a first slice containing interface elements before switching and a second slice containing interface elements after switching may be determined.
In an embodiment of the present invention, the slice transition path exhibits a chain-like structure.
Specifically, the recording actions are arranged in a chain structure, and the corresponding slice transformations are also arranged in a chain, that is, the slice transformation path is represented as a chain structure.
In a preferred embodiment of the present invention, the step S204 may specifically include the following sub-steps:
determining a common path between the slices according to the slice switching path of the chain structure; and assembling and generating branch paths among the slices as unrecorded slice conversion paths according to the mapping relation table and the common path.
Specifically, since the slice conversion paths are represented by a chain structure, after a plurality of slice conversion paths are recorded, common paths between slices are determined according to the slice conversion paths of the chain structure, then triggering is performed from the common paths, possible paths are retrieved according to the mapping relation table, and branch paths between slices are assembled and generated to serve as unrecorded slice conversion paths.
As an example, the slice switching path, as recorded in fig. 10, may be converted into a path as shown in fig. 11, in fig. 11, both paths pass through slice 1, and can be retrieved from slice 1 according to the mapping table, and the retrieval can find that, in the path of the first row, the recording operation 2 is performed on the slice 2 and then switched to the slice 3, and in the mapping relationship, the recording operation for the slice 3 includes the recording operation 3 in addition to the recording operation 2, and as can be seen from the path of the second line recorded in fig. 10, switching to slice 4 after recording action 3 on slice 3, therefore, a branch path from slice 3 in the path of the first row to slice 4 in the path of the second row can be supplemented, as an unrecorded slice switching path, as shown in fig. 12, the dashed arrow indicates a complementary slice transition path, so that the transition from slice 2 to slice 4 can be realized.
In a preferred embodiment of the present invention, the step S201 may specifically include the following sub-steps:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm to obtain a plurality of interface elements.
The preset image segmentation algorithm may be a preset algorithm for performing image segmentation, for example, a watershed algorithm. The preset image segmentation algorithm has corresponding parameters, the element granularity obtained by image segmentation is determined for different settings of the parameters, in the specific implementation, the parameters of the preset image segmentation algorithm are fixed after the parameters of the preset image segmentation algorithm are adjusted to be a proper value by training the preset image segmentation algorithm, and the preset image segmentation algorithm with the fixed parameters is adopted to carry out image segmentation on the global image corresponding to the user interface so as to obtain a plurality of interface elements.
Referring to fig. 13, a flowchart illustrating steps of another embodiment of a method for testing compatibility according to an embodiment of the present invention is shown, which may specifically include the following steps:
step S1301, carrying out image segmentation on a global image corresponding to a user interface by adopting a preset image segmentation algorithm to obtain a plurality of interface elements;
for example, the preset image segmentation algorithm may be a watershed algorithm, and the global image corresponding to the user interface may be image-segmented by the watershed algorithm to obtain a plurality of interface elements.
Step S1302, constructing a slice in a sandbox environment according to the plurality of interface elements, and establishing a mapping relation table between the slice and a recording action;
further, a slice in the sandbox environment may be constructed according to all interface elements, and a mapping relationship table between the slice and the recording action may be constructed, in the embodiment of the present invention, step S1302 is substantially similar to step S202 described above, and details are not repeated here.
Step S1303, performing the recording action on the interface elements in the user interface, and recording a slice conversion path corresponding to the recording action;
step S1303 in the embodiment of the present invention is substantially similar to step S203 described above, and is not described herein again.
Step S1304, assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
step S1304 in the embodiment of the present invention is substantially similar to step S204 described above, and is not described herein again.
Step S1305, playing back the recording action to obtain a test result;
step S1305 in the embodiment of the present invention is substantially similar to step S205 described above, and is not described herein again.
Step S1306, judging whether the preset image segmentation algorithm meets the requirements or not according to the test result;
in the embodiment of the invention, the content in all sandbox environments is formed by all recorded slice conversion paths together, all recording actions are converted into conversion of slices in the sandbox environments, and the granularity of interface elements in the slices is generated by a preset image segmentation algorithm, so that the construction quality of the sandbox environments is directly influenced by the segmentation of the interface elements. In order to improve the construction quality of the sandbox environment, in the embodiment of the invention, the preset parameters of the image segmentation algorithm can be adjusted according to the test result during playback.
Specifically, by regarding the recording playback itself as a black box, when adjusting the parameters of the preset image segmentation algorithm, the quality of the playback for the recording action is taken as a feedback to adjust the parameters of the preset image segmentation algorithm.
In a specific implementation, when some operations that cannot determine a path exist in a test result, and a corresponding recording operation cannot be performed on a slice, it is determined that a preset image segmentation algorithm does not meet requirements, and parameters of the preset image segmentation algorithm need to be adjusted. And when some operations which cannot determine the path do not exist in the test result, the preset image segmentation algorithm is considered to meet the requirement.
Step 1307, if the preset image segmentation algorithm does not meet the requirement, adjusting the parameters of the preset image segmentation algorithm according to the test result, and returning to the step of executing the image segmentation for the global image corresponding to the user interface by using the preset image segmentation algorithm to obtain a plurality of interface elements.
In the embodiment of the present invention, if the preset image segmentation algorithm does not meet the requirement, the parameters of the preset image segmentation algorithm may be adjusted according to the test result, and the step S1201 is returned to be executed until it is determined that the preset image segmentation algorithm meets the requirement.
After the preset image segmentation algorithm is judged to meet the requirements, as shown in fig. 14, when the system is stable, the parameters of the preset image segmentation algorithm can be adjusted to a proper size, and after the adjustment is finished, the parameters are fixed, and the testing work of recording and playback is completed in a sandbox environment.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 15, a block diagram of a structure of a device for testing compatibility according to an embodiment of the present invention is shown, and the device may specifically include the following modules:
the image segmentation module 1501 is configured to perform image segmentation on a global image corresponding to a user interface to obtain a plurality of interface elements;
a mapping relation table establishing module 1502, configured to establish a slice in a sandbox environment according to the multiple interface elements, and establish a mapping relation table between the slice and a recording action;
the recording module 1503 is configured to perform the recording action on an interface element in the user interface, and record a slice conversion path corresponding to the recording action;
a path assembling module 1504, configured to assemble and generate an unrecorded slice conversion path according to the mapping table and the slice conversion path;
a playback module 1505 is used for playing back the recording action to obtain the test result.
In a preferred embodiment of the present invention, the recording module 1503 includes:
the slice determining submodule is used for determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action;
a path generation submodule for generating a slice switching path from the first slice to the second slice;
and the path recording sub-module is used for recording the slice conversion path corresponding to the recording action.
In a preferred embodiment of the present invention, the recording action includes a trigger position of recording and a scene switching in the user interface, and the slice determining sub-module includes:
the image segmentation unit is used for carrying out image segmentation on the image of the user interface to obtain a plurality of interface elements;
the interface element determining unit is used for determining interface elements before switching and interface elements after switching according to the triggering position and the recorded scene switching in the user interface;
and the slice determining unit is used for determining a first slice containing the interface elements before switching and a second slice containing the interface elements after switching.
In a preferred embodiment of the invention, the slice transition path exhibits a chain-like structure.
In a preferred embodiment of the present invention, the path assembling module 1504 includes:
a common path determining submodule, configured to determine a common path between the slices according to a slice switching path of the chain structure;
and the path assembling submodule is used for assembling and generating a branch path between the slices according to the mapping relation table and the public path, and the branch path is used as an unrecorded slice conversion path.
In a preferred embodiment of the present invention, in the mapping relationship table, one slice corresponds to at least one recording action, and one recording action corresponds to at least one slice.
In a preferred embodiment of the present invention, the image segmentation module 1501 includes:
and the image segmentation submodule is used for carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm so as to obtain a plurality of interface elements.
In a preferred embodiment of the present invention, the method further comprises:
the judging module is used for judging whether the preset image segmentation algorithm meets the requirements or not according to the test result;
and the parameter adjusting module is used for adjusting the parameters of the preset image segmentation algorithm according to the test result if the preset image segmentation algorithm does not meet the requirements, and returning to execute the step of performing image segmentation on the global image corresponding to the user interface by adopting the preset image segmentation algorithm to obtain a plurality of interface elements.
In a preferred embodiment of the present invention, there is an intersection or no intersection between the interface elements included in any two slices, and the interface elements included in any two slices are not identical.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an electronic device, as shown in fig. 16, including:
a processor 1601 and a storage medium 1602, the storage medium 1602 storing machine-readable instructions executable by the processor 1601, the processor 1601 executing the machine-readable instructions to perform a method according to any one of the embodiments when the electronic device is operating. The specific implementation and technical effects are similar, and are not described herein again.
An embodiment of the present invention further provides a computer-readable storage medium, as shown in fig. 17, where a computer program 1701 is stored, and when the computer program 1701 is executed by a processor, the method according to any one of the embodiments of the present invention is executed. The specific implementation and technical effects are similar, and are not described herein again.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The method for testing compatibility and the device for testing compatibility provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (12)
1. A method of compatibility testing, comprising:
carrying out image segmentation on a global image corresponding to a user interface to obtain a plurality of interface elements;
constructing a slice in the sandbox environment according to the plurality of interface elements, and establishing a mapping relation table between the slice and the recording action;
the recording action is carried out aiming at the interface elements in the user interface, and a slice conversion path corresponding to the recording action is recorded;
assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and playing back the recording action to obtain a test result.
2. The method of claim 1, wherein recording the slice transition path corresponding to the recording action comprises:
determining a first slice containing interface elements before switching and a second slice containing interface elements after switching according to the recording action;
generating a slice transition path from the first slice to the second slice;
and recording a slice conversion path corresponding to the recording action.
3. The method of claim 2, wherein the recording action includes a trigger position for recording and a scene change in the user interface, and wherein determining the first slice including the interface element before the scene change and the second slice including the interface element after the scene change according to the recording action comprises:
carrying out image segmentation on the image of the user interface to obtain a plurality of interface elements;
determining interface elements before switching and interface elements after switching according to the triggering position and the switching of the recorded scenes in the user interface;
a first slice containing the interface elements before switching and a second slice containing the interface elements after switching are determined.
4. The method of claim 1, wherein the slice transition path exhibits a chain-like structure.
5. The method according to claim 4, wherein said assembling an unrecorded slice conversion path according to the mapping table and the slice conversion path comprises:
determining a common path between the slices according to the slice switching path of the chain structure;
and assembling and generating branch paths among the slices as unrecorded slice conversion paths according to the mapping relation table and the common path.
6. The method according to claim 1, wherein one of the slices in the mapping table corresponds to at least one of the recording actions, and one of the recording actions corresponds to at least one of the slices.
7. The method of claim 1, wherein the image segmenting the global image corresponding to the user interface to obtain the plurality of interface elements comprises:
and carrying out image segmentation on the global image corresponding to the user interface by adopting a preset image segmentation algorithm to obtain a plurality of interface elements.
8. The method of claim 7, further comprising, after the step of playing back the recorded action to obtain the test result:
judging whether the preset image segmentation algorithm meets the requirements or not according to the test result;
and if the preset image segmentation algorithm does not meet the requirements, adjusting the parameters of the preset image segmentation algorithm according to the test result, and returning to execute the step of performing image segmentation on the global image corresponding to the user interface by adopting the preset image segmentation algorithm to obtain a plurality of interface elements.
9. The method of claim 1, wherein there is an intersection or no intersection between the interface elements contained in any two of the slices, and the interface elements contained in any two of the slices are not identical.
10. An apparatus for compatibility testing, comprising:
the image segmentation module is used for carrying out image segmentation on a global image corresponding to the user interface to obtain a plurality of interface elements;
the mapping relation table establishing module is used for establishing a slice in the sandbox environment according to the plurality of interface elements and establishing a mapping relation table between the slice and the recording action;
the recording module is used for carrying out the recording action aiming at the interface elements in the user interface and recording the slice conversion path corresponding to the recording action;
the path assembling module is used for assembling and generating an unrecorded slice conversion path according to the mapping relation table and the slice conversion path;
and the playback module is used for playing back the recording action to obtain a test result.
11. An electronic device, comprising:
a processor and a storage medium storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions to perform the method of any one of claims 1-9 when the electronic device is run.
12. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674883.4A CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110674883.4A CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113392007A true CN113392007A (en) | 2021-09-14 |
| CN113392007B CN113392007B (en) | 2023-09-08 |
Family
ID=77621716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110674883.4A Active CN113392007B (en) | 2021-06-17 | 2021-06-17 | Compatibility testing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392007B (en) |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120079457A1 (en) * | 2010-09-28 | 2012-03-29 | Microsoft Corporation | Multi-path brokered test automation execution |
| US20130159784A1 (en) * | 2011-12-16 | 2013-06-20 | Sap Ag | Systems and methods for recording user interactions within a target application |
| CN103365779A (en) * | 2013-07-09 | 2013-10-23 | 广东欧珀移动通信有限公司 | Software test method and system |
| US20150169430A1 (en) * | 2013-12-13 | 2015-06-18 | International Business Machines Corporation | Selecting webpage test paths |
| US20160155466A1 (en) * | 2014-11-28 | 2016-06-02 | Sony Corporation | Information processing device, information recording medium, information processing method, and program |
| WO2017049649A1 (en) * | 2015-09-26 | 2017-03-30 | Intel Corporation | Technologies for automated application exploratory testing |
| US20180137025A1 (en) * | 2016-11-14 | 2018-05-17 | Microsoft Technology Licensing, Llc | Multi-language playback framework |
| CN108052261A (en) * | 2017-12-07 | 2018-05-18 | 广东欧珀移动通信有限公司 | Record screen method, apparatus and terminal |
| CN109189665A (en) * | 2018-08-01 | 2019-01-11 | 阿里巴巴集团控股有限公司 | Data recording, data readback and the method and device tested automatically |
| US20190095317A1 (en) * | 2017-09-25 | 2019-03-28 | Oracle International Corporation | Method and system for recording and debugging process flows |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| CN109857674A (en) * | 2019-02-27 | 2019-06-07 | 上海优扬新媒信息技术有限公司 | A kind of recording and playback test method and relevant apparatus |
| CN109871324A (en) * | 2019-01-29 | 2019-06-11 | 广州云测信息技术有限公司 | A kind of operating method and device of control |
| CN109885481A (en) * | 2019-01-14 | 2019-06-14 | 珠海金山网络游戏科技有限公司 | A kind of the recording and playback test method and device of the game NGUI based on Unity technology |
| CN110275834A (en) * | 2019-06-25 | 2019-09-24 | 中国工商银行股份有限公司 | User interface automatization test system and method |
| CN111258893A (en) * | 2020-01-13 | 2020-06-09 | 中信银行股份有限公司 | Mobile terminal application automatic testing device for randomly assembling transaction path |
| US20200233624A1 (en) * | 2019-01-17 | 2020-07-23 | EMC IP Holding Company LLC | Method, device and computer program product for updating user interface |
| US10853218B1 (en) * | 2018-12-17 | 2020-12-01 | Shutterstock, Inc. | Recording and playback of a web browser session |
| CN112579431A (en) * | 2019-09-30 | 2021-03-30 | 南京大学 | Cross-platform script recording and playback method based on image recognition |
| WO2021088724A1 (en) * | 2019-11-05 | 2021-05-14 | 深圳前海微众银行股份有限公司 | Testing method and apparatus |
| CN112817866A (en) * | 2021-02-25 | 2021-05-18 | 北京百家科技集团有限公司 | Recording playback method, device, system, computer equipment and storage medium |
-
2021
- 2021-06-17 CN CN202110674883.4A patent/CN113392007B/en active Active
Patent Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120079457A1 (en) * | 2010-09-28 | 2012-03-29 | Microsoft Corporation | Multi-path brokered test automation execution |
| US20130159784A1 (en) * | 2011-12-16 | 2013-06-20 | Sap Ag | Systems and methods for recording user interactions within a target application |
| CN103365779A (en) * | 2013-07-09 | 2013-10-23 | 广东欧珀移动通信有限公司 | Software test method and system |
| US20150169430A1 (en) * | 2013-12-13 | 2015-06-18 | International Business Machines Corporation | Selecting webpage test paths |
| US20160155466A1 (en) * | 2014-11-28 | 2016-06-02 | Sony Corporation | Information processing device, information recording medium, information processing method, and program |
| WO2017049649A1 (en) * | 2015-09-26 | 2017-03-30 | Intel Corporation | Technologies for automated application exploratory testing |
| US20180137025A1 (en) * | 2016-11-14 | 2018-05-17 | Microsoft Technology Licensing, Llc | Multi-language playback framework |
| US20190095317A1 (en) * | 2017-09-25 | 2019-03-28 | Oracle International Corporation | Method and system for recording and debugging process flows |
| CN108052261A (en) * | 2017-12-07 | 2018-05-18 | 广东欧珀移动通信有限公司 | Record screen method, apparatus and terminal |
| CN109189665A (en) * | 2018-08-01 | 2019-01-11 | 阿里巴巴集团控股有限公司 | Data recording, data readback and the method and device tested automatically |
| US10853218B1 (en) * | 2018-12-17 | 2020-12-01 | Shutterstock, Inc. | Recording and playback of a web browser session |
| CN109885481A (en) * | 2019-01-14 | 2019-06-14 | 珠海金山网络游戏科技有限公司 | A kind of the recording and playback test method and device of the game NGUI based on Unity technology |
| CN109783388A (en) * | 2019-01-16 | 2019-05-21 | 北京金山云网络技术有限公司 | UI automated testing method, device and electronic equipment |
| US20200233624A1 (en) * | 2019-01-17 | 2020-07-23 | EMC IP Holding Company LLC | Method, device and computer program product for updating user interface |
| CN109871324A (en) * | 2019-01-29 | 2019-06-11 | 广州云测信息技术有限公司 | A kind of operating method and device of control |
| CN109857674A (en) * | 2019-02-27 | 2019-06-07 | 上海优扬新媒信息技术有限公司 | A kind of recording and playback test method and relevant apparatus |
| CN110275834A (en) * | 2019-06-25 | 2019-09-24 | 中国工商银行股份有限公司 | User interface automatization test system and method |
| CN112579431A (en) * | 2019-09-30 | 2021-03-30 | 南京大学 | Cross-platform script recording and playback method based on image recognition |
| WO2021088724A1 (en) * | 2019-11-05 | 2021-05-14 | 深圳前海微众银行股份有限公司 | Testing method and apparatus |
| CN111258893A (en) * | 2020-01-13 | 2020-06-09 | 中信银行股份有限公司 | Mobile terminal application automatic testing device for randomly assembling transaction path |
| CN112817866A (en) * | 2021-02-25 | 2021-05-18 | 北京百家科技集团有限公司 | Recording playback method, device, system, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113392007B (en) | 2023-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20200314482A1 (en) | Control method and apparatus | |
| CN101989173B (en) | Image editing apparatus and image editing method | |
| CN111881019B (en) | User interface testing method and device | |
| KR102339205B1 (en) | Virtual scene display method and device, and storage medium | |
| CN109947967A (en) | Image-recognizing method, device, storage medium and computer equipment | |
| CN110288549B (en) | Video repairing method and device and electronic equipment | |
| US9449216B1 (en) | Detection of cast members in video content | |
| CN106326120B (en) | Regression testing method and device for application software and electronic equipment | |
| CN109978044B (en) | Training data generation method and device, and model training method and device | |
| CN110287877A (en) | The processing method and processing device of video object | |
| CN106339410B (en) | A kind of interactive educational resource pre-review generation method and system | |
| CN112511818B (en) | Video playing quality detection method and device | |
| CN110968808A (en) | Method and device for realizing webpage theme updating | |
| CN105117400A (en) | Information searching method and system | |
| CN115470101A (en) | Data testing method and device, computer equipment and storage medium | |
| CN116452920A (en) | Image processing method and device, storage medium and electronic equipment | |
| CN106095448B (en) | Desktop layout method and device for mobile equipment | |
| KR102086780B1 (en) | Method, apparatus and computer program for generating cartoon data | |
| US20170019490A1 (en) | Mirrored visualization of user activity in user interface | |
| CN113392007B (en) | Compatibility testing method and device | |
| CN102968244B (en) | The acquisition methods of resource pre-review figure, device and equipment | |
| CN110582020B (en) | Video generation method and device, electronic equipment and storage medium | |
| CN109255214B (en) | Authority configuration method and device | |
| CN105787067A (en) | Method and device for customizing a desktop theme | |
| CN115543168A (en) | Method, device, equipment and medium for switching application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |