CN111708711B - AB test method and device, storage medium and electronic equipment - Google Patents
AB test method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111708711B CN111708711B CN202010580963.9A CN202010580963A CN111708711B CN 111708711 B CN111708711 B CN 111708711B CN 202010580963 A CN202010580963 A CN 202010580963A CN 111708711 B CN111708711 B CN 111708711B
- Authority
- CN
- China
- Prior art keywords
- service
- app
- abtest
- module
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010998 test method Methods 0.000 title claims abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 113
- 230000008569 process Effects 0.000 claims abstract description 69
- 238000012360 testing method Methods 0.000 claims abstract description 65
- 238000012544 monitoring process Methods 0.000 claims abstract description 15
- 238000012423 maintenance Methods 0.000 abstract description 16
- 238000011161 development Methods 0.000 abstract description 15
- 230000002452 interceptive effect Effects 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000012356 Product development Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides an AB test method and device, a storage medium and electronic equipment, wherein the method comprises the following steps: executing the APP main service when the target APP is started; monitoring the execution process of the APP main service, and starting the pile module when monitoring the corresponding opportunity of the APP main service execution; acquiring a configuration item and determining client information; judging whether the client meets any execution condition or not based on the client information; if any execution condition is met, determining a target strategy corresponding to the currently met execution condition; and the stub module executes the logic code of the target strategy, and the AB test is completed when the stub module finishes executing the logic code. By applying the method, the pile module is inserted into the APP main service, and the pile module executes the logic code corresponding to the ABtest service, so that the ABtest service is abstracted into the pile module, the APP main service and the ABtest service are decoupled, the development and maintenance difficulty of the APP is reduced, and the development and maintenance efficiency is improved.
Description
Technical Field
The invention relates to the technical field of software development, in particular to an AB testing method and device, a storage medium and electronic equipment.
Background
With the rapid development of computer technology, various APPs have come up with the needs of users. In real-world product development, design and maintenance of APPs, multiple design options need to be encountered. The AB test is an optimization method of the APP, when the APP needs to be developed and designed, two or more execution schemes can be made according to the business of the APP, and the corresponding versions of each scheme are simultaneously on-line for test operation. In the process of the AB test, data such as click rate, conversion rate and the like of each version can be collected according to specified time, so that the feasibility and user experience of each version can be determined through the data, and the optimal version is selected from the versions to be put into formal use.
In the prior art, when performing an AB test, the abest service of the AB test is generally tightly coupled with the main service of the APP, and the abest service is directly executed in the process of running the main service of the APP. In the actual application process, the ABTest service generally needs to be changed along with the requirements of development and maintenance, but because the ABTest service is closely associated with the main service of the APP, the coupling degree is high, and special treatment needs to be performed on the ABTest service in the development and maintenance process, so that the difficulty of development and maintenance of the APP is increased, and the efficiency of development and maintenance of the APP is reduced.
Disclosure of Invention
In view of this, the present invention provides an AB testing method, by which a pile module can be inserted into an APP main service, and the pile module executes a logic code corresponding to an abort service, so as to decouple the APP main service from the abort service, reduce difficulty in developing and maintaining the APP, and improve efficiency of developing and maintaining.
The invention also provides an AB testing device used for ensuring the realization and the application of the method in practice.
An AB test method, comprising:
when a user starts a target APP through a client, executing an APP main service of the target APP;
calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring that the APP main service is executed to a time corresponding to an ABtest service needing AB testing;
the kernel module is applied to obtain a configuration item corresponding to the ABtest service and determine client information of the client, wherein the configuration item comprises logic codes of a plurality of strategies to be executed corresponding to the ABtest service and an execution condition corresponding to each strategy;
judging whether the client meets any execution condition in the configuration items or not based on the client information;
if the client meets any execution condition in the configuration item, determining a strategy corresponding to the execution condition currently met by the client as a target strategy;
and the pile module is applied to execute the logic code of the target strategy, and when the pile module finishes executing the logic code of the target strategy, the process that the target APP carries out AB test in the client side is completed.
The foregoing method, optionally, provides a process of inserting the stub module into the APP main business, including:
determining a life cycle of the APP main service, wherein the life cycle is a process from the beginning to the end of the execution of the APP main service;
determining the execution time and position corresponding to the ABtest service in the lifecycle, and inserting a pile module corresponding to the ABtest service into the APP main service based on the time and position, wherein the time is a specific point for starting to execute the ABtest service in the lifecycle of the APP main service, and the position refers to an area for executing the ABtest service in a page of the target APP.
The method described above, optionally, further includes:
receiving an initial configuration item corresponding to the ABtest service uploaded by a server, wherein the initial configuration item comprises a plurality of initial data, and each initial data is code data corresponding to a logic code of each strategy and condition data corresponding to an execution condition corresponding to each strategy;
determining the data type supported and executed by the APP main service, and converting each initial data in the initial configuration item into data corresponding to the data type to obtain a final configuration item corresponding to the ABtest service;
and storing the configuration items into a preset ABtest configuration source module.
Optionally, in the method, the obtaining, by the kernel module, the configuration item corresponding to the abest service includes:
and opening an access interface preset between the kernel module and the ABtest configuration source, and reading a configuration item corresponding to the ABtest service stored in the ABtest configuration source module through the access interface.
Optionally, the method for determining, based on the client information, whether the client satisfies any execution condition in the configuration item includes:
the client information is acquired to contain a plurality of information parameters, and each information parameter is respectively an equipment parameter, a client parameter, a user parameter, a time parameter and a network parameter;
determining the parameter type of each condition parameter corresponding to each execution condition, and selecting information parameters consistent with each parameter type from the client information;
matching each selected information parameter with each condition parameter corresponding to each execution condition;
and if the selected information parameters are consistent with the condition parameters corresponding to any execution condition, determining that the client side meets the execution condition.
Optionally, the method for applying the stub module to execute the logic code of the target policy includes:
determining a service type corresponding to the ABtest service, and requesting the APP main service to acquire a configuration object required by executing the service type based on the service type;
and when the APP main service returns the configuration object required by the service type, the stub module is applied to execute the logic code of the target strategy based on the configuration object.
The above method, optionally, further includes:
creating a data callback module, and associating the data callback module with the stub module;
when the stub module is applied to execute the logic code of the target policy based on the configuration object, triggering the data callback module to monitor the process of the stub module executing the logic code of the target policy, and generating log data corresponding to the process of the stub module executing the logic code of the target policy;
and when the pile module finishes executing the logic code of the target strategy, reporting the log data to a preset data server.
An AB test apparatus, comprising:
the system comprises a first execution unit, a first service execution unit and a second execution unit, wherein the first execution unit is used for executing an APP main service of a target APP when a user starts the target APP through a client;
the monitoring unit is used for calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring the opportunity corresponding to the ABtest service needing AB test executed by the APP main service;
an obtaining unit, configured to apply the kernel module to obtain a configuration item corresponding to the ABtest service, and determine client information of the client, where the configuration item includes logic codes of multiple policies to be executed corresponding to the ABtest service, and an execution condition corresponding to each policy;
a judging unit, configured to judge whether the client satisfies any execution condition in the configuration item based on the client information;
a determining unit, configured to determine, if the client satisfies any execution condition in the configuration item, that a policy corresponding to the execution condition currently satisfied by the client is a target policy;
and the second execution unit is used for executing the logic codes of the target strategy by applying the stub module and completing the process of performing the AB test on the target APP in the client when the stub module finishes executing the logic codes of the target strategy.
A storage medium comprising stored instructions, wherein the instructions, when executed, control a device on which the storage medium resides to perform the AB test method described above.
An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to perform the AB test method described above by one or more processors.
Compared with the prior art, the invention has the following advantages:
the invention provides an AB test method, which comprises the following steps: when a user starts a target APP through a client, executing an APP main service of the target APP; calling a kernel module to monitor the execution process of the APP main service in real time, and starting a pile module in the APP main service when the APP main service is monitored to be executed to the opportunity corresponding to the ABtest service needing AB testing; the application kernel module acquires a configuration item corresponding to the ABtest service, determines client information of the client and judges whether the client meets any execution condition in the configuration item; if any execution condition is met, determining a strategy corresponding to the currently met execution condition of the client as a target strategy; and the application pile module executes the logic code of the target strategy, and finishes the process of performing AB test on the target APP in the client when the pile module finishes executing the logic code of the target strategy. By applying the method provided by the invention, the pile module is inserted into the APP main service in advance, and the pile module executes the logic code corresponding to the ABtest service, so that the ABtest service is abstracted into the pile module, thereby decoupling the APP main service from the ABtest service, reducing the difficulty of development and maintenance of the APP, and improving the development and maintenance efficiency of the APP.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a method of an AB test method according to the present invention;
FIG. 2 is a flow chart of another method of the AB testing method of the present invention;
FIG. 3 is a flow chart of another method of the AB testing method according to the present invention;
FIG. 4 is a block diagram of an AB testing apparatus according to the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In this application, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions, and the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.
The invention is operational with numerous general purpose or special purpose computing device environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multi-processor apparatus, distributed computing environments that include any of the above devices or equipment, and the like.
The embodiment of the invention provides an AB test method, which can be applied to a client, and the method has a flow chart as shown in figure 1, and specifically comprises the following steps:
s101: when a user starts a target APP through a client, executing an APP main service of the target APP;
in the embodiment of the invention, the APP main service is the whole main program of the target APP, and a user can start the target APP in the client in a mechanical awakening, light touch and other modes. After receiving a starting instruction of a user to a target APP, a client executes an APP main service of the target APP, that is, starts to execute a main program of the target APP.
S102: calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring that the APP main service is executed to a time corresponding to an ABtest service needing AB testing;
in the embodiment of the invention, the main service is divided into a plurality of sub-services, each sub-service is respectively used for realizing various functions of the target APP, and the ABtest service is a sub-service which needs to be subjected to AB test in the main service and is a subprogram for testing a certain function in the target APP.
The core module is an ABtest core, is arranged in the client and is used for monitoring the execution process of the APP main service in real time, and inserting the pile module into the APP main service to realize scheduling of the pile module. Before executing the APP main service, the kernel module may be initialized in advance. The opportunity refers to a specific point in the APP main service where the abort service starts to be executed. The stub module is a plug-in unit which is arranged in the APP main service and is used for realizing the ABtest service, and a code corresponding to the ABtest service can be directly tested without any dependence item.
It should be noted that before performing AB test on the target APP, a stub module needs to be inserted into the APP main service. And when determining that the ABtest service of the AB test needs to be executed in the APP main service, starting the pile module by the kernel module, otherwise, not starting the pile module.
S103: the kernel module is applied to obtain a configuration item corresponding to the ABtest service and determine client information of the client, wherein the configuration item comprises logic codes of a plurality of strategies to be executed corresponding to the ABtest service and execution conditions corresponding to each strategy;
in the embodiment of the present invention, the configuration item corresponding to the abest service is data configured by multiple service ends, that is, includes each policy to be executed and the execution condition corresponding to each policy. The client information includes a client address, a device model of a device corresponding to the client, user information of the user, and the like. The execution condition is that a developer needs to specify condition parameters such as a device number and a client address for executing each policy when performing an AB test on the target APP. For example, the configuration item includes two policies, namely a policy a and a policy B, where the execution condition corresponding to the policy a is: the device model is the model A, the strategy A is executed, and the execution condition corresponding to the strategy B is as follows: the device model is model B execution policy B.
It should be noted that each policy in the configuration item has a logic code of an implementation process corresponding to the policy, and the logic code of each policy is a part of the abort service.
S104: judging whether the client meets any execution condition in the configuration items or not based on the client information;
in the embodiment of the present invention, the execution condition corresponding to each policy has been determined, but the client currently executing the APP main service cannot meet the set execution conditions, and whether the AB test needs to be continued is determined by determining whether the client meets any execution condition in the configuration item.
S105: if the client meets any execution condition in the configuration items, determining a strategy corresponding to the execution condition currently met by the client as a target strategy;
in the embodiment of the invention, if the client side meets any execution condition, the AB test is continued.
S106: and the pile module is applied to execute the logic code of the target strategy, and when the pile module finishes executing the logic code of the target strategy, the process that the target APP carries out AB test in the client is completed.
In the embodiment of the invention, after the target strategy which needs to be executed currently by the ABtest service is determined, the pile module executes the logic code of the target strategy, and after the pile module finishes executing the logic code of the target strategy, the pile module is jumped out, so that other functional services in the APP main service can be continuously executed, and the current AB test is completed.
In the AB testing method provided in the embodiment of the present invention, when a target APP is started, an APP main service in the target APP is executed, and an execution process of the APP main service is monitored by a kernel module. After the APP main service is executed to the opportunity of needing the AB test, starting a pile module pre-inserted into the APP main service, and acquiring a configuration item corresponding to the ABtest service when the AB test is carried out through a kernel module, so as to determine a plurality of strategies needing the AB test and corresponding execution conditions of each strategy through the configuration item. And when the client meets any execution condition in the configuration items, the stub module executes the logic code of the strategy corresponding to the execution condition so as to realize the process of performing the AB test on the client.
It should be noted that, in the APP main service, multiple stub modules may be inserted at the same time, and each stub module corresponds to multiple functions of the target APP that need to be subjected to the AB test. If a plurality of pile modules are inserted into the APP main service, the process from step S102 to step S106 is executed when the APP main service is executed to the time of each pile module, which will not be described herein again.
Further, the stub module is a plug-in inserted into the APP main service, and may isolate the APP main service from the abest service corresponding to the AB test.
Optionally, after the AB test is finished, the stub module may be deleted.
Based on the method provided by the above embodiment, a specific implementation process of the AB test method is illustrated as follows:
in the using process of the chat software a, technicians need to upgrade the chat software, wherein the display mode of the user list needs to be changed. Therefore, technicians develop two user list display modes, wherein one mode is to display a user name and a user head portrait; the other is to display only the user name. After completing the edit of the ABtest service in the two list display modes, technical personnel release the configuration items corresponding to the APP main service and the ABtest service of the chat software A. And after each client side provided with the chat software A updates the chat software A, inserting a stub module into the APP main service. When a user starts the chat software A through a client, the client executes the main APP service and monitors the execution process of the main APP service through the kernel module. When the occasion corresponding to the ABtest service is executed, the chat software A can be determined to need to display a user list, the stub module is started, the configuration item corresponding to the ABtest service is obtained, and whether the display mode of the client to the user list is to display a user name and a user head portrait or only display the user name is determined through the configuration item. And if the mode that the client needs to display the user list is to display the user name and the user head portrait, executing a logic code corresponding to the display mode, and ending the current AB test process when the user list is displayed.
By applying the method provided by the embodiment of the invention, the pile module is inserted into the APP main service in advance, and the pile module executes the logic code of the strategy corresponding to the ABtest service, so that the ABtest service is abstracted into the pile module, thereby decoupling the APP main service and the ABtest service, reducing the difficulty of development and maintenance of the APP, and improving the efficiency of development and maintenance of the APP.
In the method provided by the embodiment of the present invention, in order to ensure that the APP main service and the abest service are decoupled, a stub module needs to be inserted into the APP main service. The process of inserting the pile module may specifically include:
determining a life cycle of the APP main service, wherein the life cycle is a process from the beginning to the end of the execution of the APP main service;
determining the execution time and position corresponding to the ABtest service in the life cycle, inserting a pile module corresponding to the ABtest service into the APP main service based on the execution time and position, wherein the execution time is a specific point for starting the execution of the ABtest service in the life cycle of the APP main service, and the position refers to an area for executing the ABtest service in a page of the target APP.
In the AB test method provided in the embodiment of the present invention, the abort service is abstracted into the stub module by the lifecycle and the idea of inserting the stub module. Abstracting the ABtest service through a life cycle, and determining a specific point for executing the ABtest service and an area for executing in a page in the whole executing process according to the process from the beginning to the end of the execution of the APP main service. And inserting a pile module into the APP main service based on the corresponding time and position of the ABtest service.
It should be noted that, the pile module may be configured to perform pile insertion after determining a time and a position corresponding to the abort service according to a life cycle of the APP main service, that is, associate the pile module with the APP main service, so as to implement an execution process corresponding to the abort service through the pile module.
According to the method provided by the embodiment of the invention, on one hand, the corresponding time of the ABtest service is abstracted by using the self-defined life cycle, and the method can be suitable for various App main services. Any service has a life cycle of itself, and pile modules can be conveniently inserted at the time points only by defining the time of the life cycle according to the actual situation. The points at each opportunity can be self-defined, have very high flexibility, and the readability is high to be convenient for later maintenance. On the other hand, through the life cycle and the pile inserting technology, a plurality of ABtest services can be abstracted to corresponding plug-ins respectively, so that the coupling of the ABtest services and the main service of the App is low, the expandability is high, and all the plug-ins are based on the same abstraction, and the uniform management is facilitated.
In the method provided in the embodiment of the present invention, before performing the AB test, data corresponding to each policy that needs to be executed needs to be processed to ensure uniform management during later development and maintenance, and a process of processing the configuration item is shown in fig. 2, and may specifically include:
s201: receiving an initial configuration item corresponding to the ABtest service uploaded by a server, wherein the initial configuration item comprises a plurality of initial data, and each initial data is code data corresponding to a logic code of each strategy and condition data corresponding to an execution condition corresponding to each strategy;
in the embodiment of the invention, a technician configures the configuration item of the target APP at the server, and the configuration item configured at the server is an initial configuration item. And the server uploads the initial configuration item to each client in the test range according to the test range of the AB test set by a technician.
Each code data in the configuration item may be a code file containing a logic code of the corresponding policy, and each condition data contains a parameter file of each condition parameter of the corresponding policy.
S202: determining the data types supported and executed by the APP main service, and converting each initial data in the initial configuration items into data corresponding to the data types to obtain final configuration items corresponding to the ABtest service;
in the embodiment of the present invention, in the process of performing the AB test, if the abest service needs to be executed, the configuration item corresponding to the abest service needs to be a configuration item that can be identified by the APP, so that each initial data in the initial configuration item is converted into data that can be identified by the APP according to the data type supported by the main service of the APP, and final data is obtained.
It should be noted that, the data conversion process may be completed by a kernel module in the client or may be completed by the abest configuration source module.
S203: and storing the configuration items into a preset ABtest configuration source module.
In the embodiment of the present invention, the ABtest configuration Source may be an application Data Source technology, and synchronizes configuration items corresponding to ABtest services configured by multiple servers to a local and converts the configuration items into a storage module stored in a uniform format. Therefore, after the conversion of the data format is performed, the configuration item is stored into the ABtest configuration source.
In the AB test method provided in the embodiment of the present invention, if the AB test of the target APP needs to be performed, the configuration item needs to be set in advance. Technical staff well configure configuration items corresponding to the ABtest service through the server and upload the configuration items to the client through the server, and the client converts data types of all data in the configuration items uploaded by the server according to data types supported by the target APP so as to ensure that the data in the configuration items can meet the execution requirements of the target APP when the AB test is carried out. And storing the configuration item corresponding to the converted data in the ABtest configuration source, and acquiring the configuration item from the ABtest configuration source when the ABtest service needs to be executed so as to realize the execution process of the ABtest service. The method provided by the embodiment of the invention can convert each data in the configuration item into the data type supported by the target APP, so that the subsequent calling of the configuration item is facilitated, and the management of each data in the configuration item is facilitated.
In the present invention, after storing the configuration item in the abest configuration source, when starting the target APP and executing the APP main service to the time corresponding to the abest service, the configuration item needs to be obtained from the abest configuration source, and the specific process may include:
and opening an access interface preset between the kernel module and the ABtest configuration source, and reading a configuration item corresponding to the ABtest service stored in the ABtest configuration source module through the access interface.
In the AB test method provided by the embodiment of the present invention, the configuration items are stored in the attest configuration source, which is convenient for maintaining and managing each data in the configuration items. And when the APP main service of the target APP is executed to the time corresponding to the ABtest service, starting an access interface between the kernel module and the ABtest configuration source, wherein the access interface can be an API (application program interface). The data in the ABtest configuration source can be directly read through the access interface, namely, the configuration item corresponding to the ABtest service is read.
By applying the method provided by the embodiment of the invention, the data in the ABtest configuration source is read through the uniform access interface, so that the subsequent uniform management can be facilitated, and the development and maintenance cost can be saved.
The method provided in the embodiment of the present invention, based on the content in the step S104 and according to the client information of the client, determines whether the client meets any execution condition in the configuration item, and specifically includes:
the client information is acquired to contain a plurality of information parameters, and each information parameter is respectively an equipment parameter, a client parameter, a user parameter, a time parameter and a network parameter;
determining the parameter type of each condition parameter corresponding to each execution condition, and selecting information parameters consistent with each parameter type from the client information;
matching each selected information parameter with each condition parameter corresponding to each execution condition;
and if the selected information parameters are consistent with the condition parameters corresponding to any execution condition, determining that the client meets the execution condition.
In the AB testing method provided in the embodiment of the present invention, the client information includes a plurality of information parameters, which respectively include a device parameter, a client parameter, a user parameter, a time parameter, and a network parameter. The device parameter may be a device number, a device version, etc. of a device currently used by the user; the client parameter can be the client number, the client address and the like of the client; the user parameters may be the name, age, contact, etc. of the user; the time parameter may be a current time period; the network parameter may be a network number of the client's current connectionless network. In addition, each execution condition may include a plurality of condition parameters, and each condition parameter may be a device condition parameter, a client condition parameter, a user condition parameter, a time condition parameter, and a network condition parameter that specify a policy corresponding to the execution condition. Each condition parameter corresponds to a parameter type, and the information types with the same parameter types are selected from the client information according to the parameter types of the condition parameters corresponding to the execution conditions.
For example, if each condition parameter of each execution condition is an equipment condition parameter and a network condition parameter, the parameter type corresponding to each execution condition is determined to be an equipment parameter type and a network parameter type. And selecting the device condition parameters and the network condition parameters corresponding to each execution condition respectively from the device parameters and the network parameters in the client information. And if the equipment parameter and the network parameter are consistent with the equipment condition parameter and the network condition parameter corresponding to any one execution condition, determining that the client meets the execution condition.
It should be noted that, the setting of the execution condition corresponding to each policy mainly sets the relevant condition parameters according to the application scenarios of the target APP and the abort service, and in each condition parameter corresponding to each execution condition, there may be an execution condition with a part of condition parameters being consistent.
The method provided by the invention can determine the logic code corresponding to the target strategy to be executed at the client according to each information parameter and condition parameter, and ensure the successful execution of the ABtest service.
In the method provided in the embodiment of the present invention, based on the content of step S106, after determining the target policy, the stub module may be applied to execute the corresponding logic code, and a process of specifically applying the stub module to execute the logic code of the target policy is shown in fig. 3, which may include:
s301: determining a service type corresponding to the ABtest service, and requesting the APP main service to acquire a configuration object required by executing the service type based on the service type;
in the embodiment of the present invention, the service type of the abort service may be abstracted into three services, which may specifically include: and displaying the service, the interactive service and the internal service. The exhibition service comprises the following steps: in view of the abstract of the presentation service, different UIs are usually presented in a certain page of the App according to configuration items of the ABTest service, and according to the abstract, only the main service of the App is required to provide a container view. The interactive service comprises the following steps: according to the abstraction, only the App main service is required to provide an interaction component required by the ABtest service, namely, the specific interaction process between the interaction component and the user can be realized by the stub module; the internal service: the method is characterized in that a user triggers or an App main service runs to a certain time, some services which are not sensed by the user need to be processed, the non-sensed services are different from display services and interactive services, such as a service request interface and the like, and similar to the interactive services, the process which is not sensed by the user can be realized by a pile module only by the App main service providing service configuration such as data, objects and the like required by the ABTest service.
It is understood that the service configurations of the container view, the interactive components, the data and the objects are all configuration objects. Each service type corresponds to at least one configuration object, and the APP main service is requested to acquire the corresponding configuration object according to the service type corresponding to the ABtest service.
S302: and when the APP main service returns the configuration object required by the service type, the stub module is applied to execute the logic code of the target strategy based on the configuration object.
In embodiments of the present invention, the stub module requires logic code that relies on the configuration object to execute the target policy.
For example, if the service type is a show service, the pile module can realize the show UIs of different ABTest services by only needing the App main service to provide a container view; if the service type is interactive service, only the App main service is needed to provide the interactive component required by the ABtest service, and the pile module can realize the specific interactive process between the interactive component and the user; if the service type is an internal service, the pile module can realize the process of no perception of the user only by the service configuration of data, objects and the like required by the ABTest service provided by the App main service.
In the AB testing method provided in the embodiment of the present invention, in the process of implementing the abest service by the stub module, the corresponding logic code needs to be executed according to the target policy, and since the abest service includes different service types, the stub module can actively acquire the configuration object corresponding to the service type corresponding to the abest service from the APP main service. The APP main business provides corresponding configuration objects according to business types, and the stub module realizes logic codes of the target strategy depending on the configuration objects returned by the APP main business.
It should be noted that the stub module cannot sense the time for specifically starting the stub module in the lifecycle corresponding to the APP main service. But the APP main service can know whether the current time in the corresponding life cycle of the APP main service needs to start the stub module to execute the logic code corresponding to the ABtest service. And when determining that the stub module needs to configure the object, the APP main service returns the configuration object to the stub module according to the request of the stub module, so that the stub module can execute the logic code depending on the configuration object.
By applying the method provided by the embodiment of the invention, the stub module obtains the corresponding configuration object by combining with the service type corresponding to the ABtest service, so as to ensure that the execution process of the ABtest service is isolated from the APP main service, the APP main service and the ABtest service are decoupled, and the corresponding logic code can be successfully executed.
In the method provided by the embodiment of the present invention, after a pile module is inserted into an APP main service, the APP main service and an abest service can be decoupled, and at this time, to obtain execution data related to each policy in an AB test process, it is necessary to call back log data related to the AB test each time, and the specific process is as follows:
creating a data callback module and associating the data callback module with the stub module;
when the stub module is applied to execute the logic code of the target policy based on the configuration object, triggering the data callback module to monitor the process of the stub module executing the logic code of the target policy, and generating log data corresponding to the process of the stub module executing the logic code of the target policy;
and when the pile module finishes executing the logic code of the target strategy, reporting the log data to a preset data server.
In the AB testing method provided in the embodiment of the present invention, when the APP main service is inserted into the stub module, a data callback module may be created at the same time, and the data callback module has monitoring and callback functions. When the pile module is applied to execute the logic codes, the data callback module monitors the execution process of the pile module in real time, generates corresponding log data, and uploads the log data to the data server side by the callback module when the pile module finishes executing the logic codes of the target strategy.
The data server is a server for counting data such as click rate, conversion rate and the like of each strategy according to log data. The data server may be a server that uploads the initial configuration item in the above embodiment.
By applying the method provided by the invention, the related log data of each AB test can be acquired in a data callback mode so as to determine the feasibility of each strategy, and technicians can select the optimal strategy from all the strategies.
The specific implementation procedures and derivatives thereof of the above embodiments are within the scope of the present invention.
Corresponding to the method described in fig. 1, an AB testing apparatus provided in an embodiment of the present invention is further provided for specifically implementing the method in fig. 1, and the AB testing apparatus provided in the embodiment of the present invention may be applied to a computer terminal or various mobile devices, and a schematic structural diagram of the AB testing apparatus is shown in fig. 4, and specifically includes:
a first execution unit 401, configured to execute an APP main service of a target APP when a user starts the target APP through a client;
a monitoring unit 402, configured to call a preset kernel module to monitor an execution process of the APP main service in real time, and start a pile module pre-inserted into the APP main service when monitoring that the APP main service is executed to a time corresponding to an abest service that needs to be subjected to an AB test;
an obtaining unit 403, configured to apply the kernel module to obtain a configuration item corresponding to the attest service, and determine client information of the client, where the configuration item includes logic codes of multiple policies to be executed corresponding to the attest service, and an execution condition corresponding to each policy;
a determining unit 404, configured to determine, based on the client information, whether the client satisfies any execution condition in the configuration item;
a determining unit 405, configured to determine, if the client satisfies any execution condition in the configuration item, that a policy corresponding to the execution condition currently satisfied by the client is a target policy;
a second executing unit 406, configured to apply the stub module to execute the logic code of the target policy, and when the stub module finishes executing the logic code of the target policy, complete a process of performing an AB test on the target APP in the client.
In the device provided by the embodiment of the invention, when the target APP is started, the first execution unit executes the main APP service of the target APP, the monitoring unit monitors the execution process of the main APP service, and when a specific time is reached, the pile module is started. The obtaining unit applies the kernel module to obtain the configuration item corresponding to the ABtest service, the client information is sent to the client, the judging unit judges whether the client meets any execution condition in the configuration item according to the client information, and after the client meets any execution condition, the determining unit determines that the strategy corresponding to the execution condition in the configuration item is the target strategy. Finally, the second execution unit application stub executes the logic code of the target policy to implement the process of the AB test.
By applying the device provided by the invention, the pile module can be inserted into the APP main service, and the pile module executes the logic code corresponding to the ABtest service, so that the ABtest service is abstracted into the pile module, the APP main service and the ABtest service are decoupled, the development and maintenance difficulty of the APP is reduced, and the development and maintenance efficiency is improved.
The device provided by the embodiment of the invention further comprises:
the instrumentation unit is used for determining a life cycle of the APP main service, wherein the life cycle is a process from the beginning to the end of the execution of the APP main service; determining the execution time and position corresponding to the ABtest service in the lifecycle, and inserting a pile module corresponding to the ABtest service into the APP main service based on the time and position, wherein the time is a specific point for starting to execute the ABtest service in the lifecycle of the APP main service, and the position refers to an area for executing the ABtest service in a page of the target APP.
The device provided by the embodiment of the invention further comprises:
the data conversion unit is used for receiving an initial configuration item corresponding to the ABtest service uploaded by a server, wherein the initial configuration item comprises a plurality of initial data, and each initial data is code data corresponding to a logic code of each strategy and condition data corresponding to an execution condition corresponding to each strategy; determining the data type supported and executed by the APP main service, and converting each initial data in the initial configuration item into data corresponding to the data type to obtain a final configuration item corresponding to the ABtest service; and storing the configuration items into a preset ABtest configuration source module.
In the apparatus provided in the embodiment of the present invention, the obtaining unit 403 includes:
and the first acquiring subunit is used for starting a preset access interface between the kernel module and the ABtest configuration source, and reading the configuration item corresponding to the ABtest service stored in the ABtest configuration source module through the access interface.
In the apparatus provided in the embodiment of the present invention, the determining unit 404 includes:
the second acquiring subunit is configured to acquire that the client information includes a plurality of information parameters, where each information parameter is an equipment parameter, a client parameter, a user parameter, a time parameter, and a network parameter;
a first determining subunit, configured to determine a parameter type of each condition parameter corresponding to each execution condition, and select an information parameter that is consistent with each parameter type from the client information;
a matching subunit, configured to match each selected information parameter with each condition parameter corresponding to each execution condition;
and the second determining subunit is configured to determine that the client satisfies the execution condition if each selected information parameter is consistent with each condition parameter corresponding to any execution condition.
In the apparatus provided in the embodiment of the present invention, the second executing unit 406 includes:
a third determining subunit, configured to determine a service type corresponding to the abest service, and request the APP main service to acquire a configuration object required for executing the service type based on the service type;
and the execution subunit is configured to, when the APP main service returns the configuration object required by the service type, apply the stub module to execute the logic code of the target policy based on the configuration object.
The device provided by the embodiment of the invention further comprises:
the association unit is used for creating a data callback module and associating the data callback module with the stub module;
the triggering unit is used for triggering the data callback module to monitor the process of the stub module executing the logic code of the target policy and generate log data corresponding to the process of the stub module executing the logic code of the target policy when the stub module is applied to execute the logic code of the target policy based on the configuration object;
and the callback unit is used for reporting the log data to a preset data server when the pile module finishes executing the logic code of the target strategy.
The specific working processes of each unit and sub-unit in the AB test apparatus disclosed in the above embodiment of the present invention can be referred to the corresponding contents in the AB test method disclosed in the above embodiment of the present invention, and are not described herein again.
The embodiment of the invention also provides a storage medium, which comprises a stored instruction, wherein when the instruction runs, the device where the storage medium is located is controlled to execute the AB test method.
An embodiment of the present invention further provides an electronic device, a schematic structural diagram of which is shown in fig. 5, and the electronic device specifically includes a memory 501 and one or more instructions 502, where the one or more instructions 502 are stored in the memory 501, and are configured to be executed by one or more processors 503 to perform the following operations for the one or more instructions 502:
when a user starts a target APP through a client, executing an APP main service of the target APP;
calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring that the APP main service is executed to a time corresponding to an ABtest service needing AB testing;
the kernel module is applied to obtain a configuration item corresponding to the ABtest service and determine client information of the client, wherein the configuration item comprises logic codes of a plurality of strategies to be executed corresponding to the ABtest service and execution conditions corresponding to each strategy;
judging whether the client meets any execution condition in the configuration items or not based on the client information;
if the client meets any execution condition in the configuration items, determining a strategy corresponding to the execution condition currently met by the client as a target strategy;
and the pile module is applied to execute the logic code of the target strategy, and when the pile module finishes executing the logic code of the target strategy, the process that the target APP carries out AB test in the client is completed.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments, which are substantially similar to the method embodiments, are described in a relatively simple manner, and reference may be made to some descriptions of the method embodiments for relevant points. The above-described system and system embodiments are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement without inventive effort.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both.
To clearly illustrate this interchangeability of hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. An AB test method, comprising:
when a user starts a target APP through a client, executing an APP main service of the target APP;
calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring that the APP main service is executed to a time corresponding to an ABtest service needing AB testing;
the kernel module is applied to obtain a configuration item corresponding to the ABtest service and determine client information of the client, wherein the configuration item comprises logic codes of a plurality of strategies to be executed corresponding to the ABtest service and an execution condition corresponding to each strategy;
judging whether the client meets any execution condition in the configuration items or not based on the client information;
if the client meets any execution condition in the configuration items, determining a strategy corresponding to the execution condition currently met by the client as a target strategy;
and the pile module is applied to execute the logic code of the target strategy, and when the pile module finishes executing the logic code of the target strategy, the process that the target APP carries out AB test in the client side is completed.
2. The method of claim 1, wherein the process of inserting the stub module in the APP master business comprises:
determining a life cycle of the APP main service, wherein the life cycle is a process from the beginning to the end of the execution of the APP main service;
determining the execution time and position corresponding to the ABtest service in the lifecycle, and inserting a pile module corresponding to the ABtest service into the APP main service based on the time and position, wherein the time is a specific point for starting to execute the ABtest service in the lifecycle of the APP main service, and the position refers to an area for executing the ABtest service in a page of the target APP.
3. The method of claim 1, further comprising:
receiving an initial configuration item corresponding to the ABtest service uploaded by a server, wherein the initial configuration item comprises a plurality of initial data, and each initial data is code data corresponding to a logic code of each strategy and condition data corresponding to an execution condition corresponding to each strategy;
determining the data types supported and executed by the APP main service, and converting each initial data in the initial configuration items into data corresponding to the data types to obtain final configuration items corresponding to the ABtest service;
and storing the configuration items into a preset ABtest configuration source module.
4. The method according to any one of claims 1 or 3, wherein the applying the kernel module to obtain the configuration item corresponding to the ABtest service comprises:
and opening an access interface preset between the kernel module and the ABtest configuration source, and reading a configuration item corresponding to the ABtest service stored in the ABtest configuration source module through the access interface.
5. The method according to claim 1, wherein the determining whether the client satisfies any execution condition in the configuration items based on the client information comprises:
the client information is acquired to contain a plurality of information parameters, and each information parameter is respectively an equipment parameter, a client parameter, a user parameter, a time parameter and a network parameter;
determining the parameter type of each condition parameter corresponding to each execution condition, and selecting information parameters consistent with each parameter type from the client information;
matching each selected information parameter with each condition parameter corresponding to each execution condition;
and if the selected information parameters are consistent with the condition parameters corresponding to any execution condition, determining that the client side meets the execution condition.
6. The method of claim 1, wherein the applying the stub module to execute the logic code of the target policy comprises:
determining a service type corresponding to the ABtest service, and requesting the APP main service to acquire a configuration object required by executing the service type based on the service type;
and when the APP main service returns the configuration object required by the service type, the stub module is applied to execute the logic code of the target strategy based on the configuration object.
7. The method of claim 6, further comprising:
creating a data callback module and associating the data callback module with the stub module;
when the stub module is applied to execute the logic code of the target policy based on the configuration object, triggering the data callback module to monitor the process of the stub module executing the logic code of the target policy, and generating log data corresponding to the process of the stub module executing the logic code of the target policy;
and when the pile module finishes executing the logic code of the target strategy, reporting the log data to a preset data server.
8. An AB test apparatus, comprising:
the system comprises a first execution unit, a first service execution unit and a second execution unit, wherein the first execution unit is used for executing an APP main service of a target APP when a user starts the target APP through a client;
the monitoring unit is used for calling a preset kernel module to monitor the execution process of the APP main service in real time, and starting a pile module pre-inserted into the APP main service when monitoring the opportunity corresponding to the ABtest service needing AB test executed by the APP main service;
an obtaining unit, configured to apply the kernel module to obtain a configuration item corresponding to the ABtest service, and determine client information of the client, where the configuration item includes logic codes of multiple policies to be executed corresponding to the ABtest service, and an execution condition corresponding to each policy;
a judging unit, configured to judge whether the client satisfies any execution condition in the configuration item based on the client information;
a determining unit, configured to determine, if the client satisfies any execution condition in the configuration item, that a policy corresponding to the execution condition currently satisfied by the client is a target policy;
and the second execution unit is used for executing the logic code of the target strategy by applying the stub module and completing the process of performing the AB test on the target APP in the client when the stub module finishes executing the logic code of the target strategy.
9. A storage medium comprising stored instructions, wherein the instructions, when executed, control a device on which the storage medium resides to perform an AB test method as claimed in any one of claims 1 to 7.
10. An electronic device comprising a memory, and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by the one or more processors to perform the AB test method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010580963.9A CN111708711B (en) | 2020-06-23 | 2020-06-23 | AB test method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010580963.9A CN111708711B (en) | 2020-06-23 | 2020-06-23 | AB test method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111708711A CN111708711A (en) | 2020-09-25 |
| CN111708711B true CN111708711B (en) | 2023-04-07 |
Family
ID=72542253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010580963.9A Active CN111708711B (en) | 2020-06-23 | 2020-06-23 | AB test method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111708711B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69814078D1 (en) * | 1998-10-15 | 2003-06-05 | Halmon Laboratoira Beheer B V | Improved method for determining allergies |
| CN103246597A (en) * | 2012-02-07 | 2013-08-14 | 腾讯科技（深圳）有限公司 | Parameter testing method and device |
| CN107092555A (en) * | 2016-08-05 | 2017-08-25 | 北京小度信息科技有限公司 | A kind of method and system of test application program |
| CN108762846A (en) * | 2018-05-30 | 2018-11-06 | 努比亚技术有限公司 | Plug-in unit real-time recommendation method, server and computer readable storage medium |
| CN109101425A (en) * | 2018-08-14 | 2018-12-28 | 阿里巴巴集团控股有限公司 | A kind of index of the page AB test of mobilism buries point methods and device |
| CN109471795A (en) * | 2018-10-16 | 2019-03-15 | 平安普惠企业管理有限公司 | It is grouped test method, device, computer equipment and storage medium |
| CN109753424A (en) * | 2017-11-06 | 2019-05-14 | 北京京东尚科信息技术有限公司 | The method and apparatus of AB test |
| CN109948016A (en) * | 2017-10-31 | 2019-06-28 | 北京嘀嘀无限科技发展有限公司 | Application message method for pushing, device, server and computer readable storage medium |
| CN110096396A (en) * | 2019-03-18 | 2019-08-06 | 北京奇艺世纪科技有限公司 | It is grouped control method, distribution application system and the storage medium of test |
| CN110601922A (en) * | 2019-09-18 | 2019-12-20 | 北京三快在线科技有限公司 | Method and device for realizing comparison experiment, electronic equipment and storage medium |
| CN110765182A (en) * | 2019-10-29 | 2020-02-07 | 北京达佳互联信息技术有限公司 | Data statistical method and device, electronic equipment and storage medium |
-
2020
- 2020-06-23 CN CN202010580963.9A patent/CN111708711B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69814078D1 (en) * | 1998-10-15 | 2003-06-05 | Halmon Laboratoira Beheer B V | Improved method for determining allergies |
| CN103246597A (en) * | 2012-02-07 | 2013-08-14 | 腾讯科技（深圳）有限公司 | Parameter testing method and device |
| CN107092555A (en) * | 2016-08-05 | 2017-08-25 | 北京小度信息科技有限公司 | A kind of method and system of test application program |
| CN109948016A (en) * | 2017-10-31 | 2019-06-28 | 北京嘀嘀无限科技发展有限公司 | Application message method for pushing, device, server and computer readable storage medium |
| CN109753424A (en) * | 2017-11-06 | 2019-05-14 | 北京京东尚科信息技术有限公司 | The method and apparatus of AB test |
| CN108762846A (en) * | 2018-05-30 | 2018-11-06 | 努比亚技术有限公司 | Plug-in unit real-time recommendation method, server and computer readable storage medium |
| CN109101425A (en) * | 2018-08-14 | 2018-12-28 | 阿里巴巴集团控股有限公司 | A kind of index of the page AB test of mobilism buries point methods and device |
| CN109471795A (en) * | 2018-10-16 | 2019-03-15 | 平安普惠企业管理有限公司 | It is grouped test method, device, computer equipment and storage medium |
| CN110096396A (en) * | 2019-03-18 | 2019-08-06 | 北京奇艺世纪科技有限公司 | It is grouped control method, distribution application system and the storage medium of test |
| CN110601922A (en) * | 2019-09-18 | 2019-12-20 | 北京三快在线科技有限公司 | Method and device for realizing comparison experiment, electronic equipment and storage medium |
| CN110765182A (en) * | 2019-10-29 | 2020-02-07 | 北京达佳互联信息技术有限公司 | Data statistical method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111708711A (en) | 2020-09-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11816492B2 (en) | Third party integration of plugins and widgets | |
| CN113645071B (en) | Cluster deployment method, system, medium and electronic terminal | |
| US8660833B2 (en) | Method, computer program product and apparatus for providing an interactive network simulator | |
| WO2020015191A1 (en) | Business rule releasing and managing method, electronic device and readable storage medium | |
| CN111142884B (en) | Version deployment method and device of applet, electronic equipment and storage medium | |
| CN110955600B (en) | Interface testing method and device | |
| CN109359020A (en) | Start time test method and device, computer installation and storage medium | |
| CN109684188B (en) | Test method and device | |
| CN111352836A (en) | Pressure testing method and related device | |
| EP3567600B1 (en) | Improving a runtime environment for imaging applications on a medical device | |
| CN109739704A (en) | A kind of interface test method, server-side and computer readable storage medium | |
| US9971611B2 (en) | Monitoring a mobile device application | |
| CN116414370A (en) | Platform construction method and device based on low codes, medium and electronic equipment | |
| CN106952426B (en) | Data processing method and device | |
| Dobrica et al. | Experiencing native mobile health applications development | |
| CN111612135B (en) | Method and device for information interaction | |
| CN111523676B (en) | Method and device for assisting machine learning model to be online | |
| CN111708711B (en) | AB test method and device, storage medium and electronic equipment | |
| KR20120102967A (en) | System and method for verifying application | |
| CN112559124A (en) | Model management system and target operation instruction processing method and device | |
| US10073689B2 (en) | Managing application lifecycles within a federation of distributed software applications | |
| CN116662132A (en) | Evaluation method, virtual deployment method, computer device, and storage medium | |
| CN111488268A (en) | Dispatching method and dispatching device for automatic test | |
| CN115016775A (en) | Interface simulation method and device, storage medium and electronic equipment | |
| CN111880996B (en) | Bare metal data acquisition method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |