CN110008131B - Method and device for managing area AB experiment based on algorithm - Google Patents
Method and device for managing area AB experiment based on algorithm Download PDFInfo
- Publication number
- CN110008131B CN110008131B CN201910294985.6A CN201910294985A CN110008131B CN 110008131 B CN110008131 B CN 110008131B CN 201910294985 A CN201910294985 A CN 201910294985A CN 110008131 B CN110008131 B CN 110008131B
- Authority
- CN
- China
- Prior art keywords
- experiment
- configuration
- application
- user
- api
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method and a device for managing an area AB experiment based on an algorithm; the method comprises the following steps: creating an experiment, receiving initial experiment configuration and generating an experiment ID; when the application is started, the SDK is accessed to each language, the configuration of all experiment IDs used by the application configuration is pulled through the API and stored in a file system, and new experiment configuration is periodically pulled from the API; according to the user access information, performing experimental calculation, returning to the access application whether the scheme A or the scheme B is currently used, and recording and storing the scheme used by the user ID; sending the recorded information to a data analysis platform through a log collection system, and displaying an analysis result and experimental index data through a visual platform; after the function is on line, a developer does not need to intervene operation, one application supports a plurality of experiments to be carried out in parallel, the plurality of applications can share the same experiment to be carried out, and the service cannot be influenced by the fault of the API system.
Description
Technical Field
The invention relates to the technical field of online AB test experiments, in particular to a method and a device for managing an area AB experiment based on an algorithm.
Background
the other scheme is that different processing is carried out on a load balancing layer, the AB is respectively issued by using different application versions, the scheme has the problems that two versions need to be operated, operation and maintenance are required to carry out online change operation, different flow ratios are allocated, and finally, online operation and maintenance change is required to be carried out for issuing, so that the AB experiment cost is high.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a method and a device for managing an area AB experiment based on an algorithm.
In one aspect, the invention provides a method for managing an area AB experiment based on an algorithm, the method comprising the steps of:
Creating an experiment, receiving initial experiment configuration and generating an experiment ID;
When the application is started, the SDK is accessed to each language, the configuration of all experiment IDs used by the application configuration is pulled through the API and stored in a file system, and after the application is started, new experiment configuration is periodically pulled from the API;
According to the user access information, performing experimental calculation, returning to the access application whether the scheme A or the scheme B is currently used, and recording and storing the scheme used by the user ID;
And sending the recorded information to a data analysis platform through a log collection system, and displaying the analysis result and the experimental index data through a visual platform.
Optionally, the initial experiment configuration includes a user session or a login account ID of the user, an application area for configuring the experiment, a user traffic ratio used by the experiment, and an AB option weight ratio.
Optionally, before the configuration of all experiment IDs used by the application configuration is stored in the file system through the API, the method further includes: the API acquires the initial experiment configuration and compiles the initial experiment configuration into configuration files in various formats respectively; the API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request; the API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, and the subsequent SDK request needs to be provided with an ETag value on the subsequent request of the SDK.
Optionally, the step of pulling, by each language access SDK, the latest configuration of all configured experiment IDs through the API when the application is started, and periodically pulling a new experiment configuration from the API, includes: when the initial downloading of all configured experiment IDs fails, the application is failed to start; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file; if the follow-up updating check is successful, the original configuration is covered; if the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
Optionally, the step of performing experimental calculation according to the user access identity information includes: transmitting a user identity ID, a user IP and a user session ID; judging whether the current access user is in a configured experimental area or not based on the IP of the user; if the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording; if the test area is in the test area, entering an experimental calculation stage.
In another aspect, the present invention provides an apparatus for managing an area AB experiment implemented based on an algorithm, the apparatus comprising:
The experiment creating unit is used for creating an experiment, receiving initial experiment configuration and generating an experiment ID;
The method comprises the steps that an experiment configuration obtaining unit is used for pulling all the configuration of experiment IDs used by application configuration through an API and storing the configuration in a file system when the application of each language access SDK is started, and after the application is started, new experiment configuration is periodically pulled from the API;
The experiment calculation storage unit is used for calculating an experiment according to the user access information, returning the experiment to the access application to use the scheme A or the scheme B currently, and recording and storing the scheme used by the user ID;
And the data analysis and display unit is used for sending the recorded information to the data analysis platform through the log collection system and displaying the analysis result and the experimental index data through the visual platform.
Optionally, the experiment creating unit is further configured to obtain a login account ID of the user session or the user, configure an application area of the experiment, a user traffic ratio used in the experiment, and an AB option weight ratio.
Optionally, the acquiring experiment configuration unit is further configured to acquire the initial experiment configuration through an API, and compile the initial experiment configuration into configuration files in multiple formats respectively; the API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request; the API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, and the subsequent SDK request needs to be provided with an ETag value on the subsequent request of the SDK.
Optionally, the acquiring experiment configuration unit is further configured to fail to start the application when the initial downloading of all configured experiment IDs fails; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file; if the follow-up updating check is successful, the original configuration is covered; if the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
Optionally, the experiment calculation storage unit is further configured to transmit a user identity ID, an IP of the user, and a session ID of the user; judging whether the current access user is in a configured experimental area or not based on the IP of the user; if the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording; if the test area is in the test area, entering an experimental calculation stage.
The invention has the beneficial effects that:
In the invention, through managing the background and managing the experiment in unison, the authority configuration is convenient for the personnel in the same department to manage the experiment cooperatively; developers only need to intervene in an experiment access stage, after the functions are on-line, the developers and operation and maintenance personnel do not need to intervene in operation, the operation can be completely carried out by products or operators, the experiment range is expanded, and finally, release and selection A or B do not need to be applied to release on-line; the experiment can be based on the user, and also can be based on the access session of the user, and the user always uses A or B under the same user or the same session, so that the consistency of experience is ensured. One application supports a plurality of experiments to be carried out in parallel, the same page supports a plurality of experiments to be carried out simultaneously, and the plurality of applications can share the same experiment to be carried out. The application range of the experiment can be controlled, the experiment can be performed only in a certain area, the influence surface is reduced, and the experiment can be stopped at any time to continue using the original logic. The computing logic is dispersed to each SDK for processing, and the dispersion of computing resources can not affect the management background or the API. And the SDK access of a plurality of languages is supported, so that the front-end and back-end operation and maintenance line products can be accessed and used for AB test.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
Fig. 1 is a schematic flow chart of a method for managing an area AB experiment based on algorithm implementation according to an embodiment of the present invention;
Fig. 2 is a schematic structural diagram of a device for area AB experimental management based on algorithm implementation according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
in an on-line AB test experiment, the current general AB experiment mode is to use the function of a flow statistics platform to make two pages of different UR L, and to make statistics summary of the effect according to different UR L, the scheme has the problems that the page displayed by a user can not be controlled, the flexibility is not strong, and only AB test can be made based on the whole page, after the AB test is completed, the function needs to be re-adjusted to be on-line or page jump is made, when the number of pages needing to jump in one application is more, the management is troublesome.
The specific embodiment of the invention provides a method for managing an area AB experiment based on an algorithm, which is shown in figure 1 and comprises the following steps:
In step S101, an experiment is created, an initial experiment configuration is received, and an experiment ID is generated.
In the embodiment of the invention, a product operator creates an experiment through a management background, configures an application area of the experiment based on user conversation or user identity in initial experiment configuration, configures the flow of the area for the first time to carry out the experiment, and configures the weight of the part AB entering the experiment; thereby generating an experiment ID.
In step S102, when the application is started, the language access SDK pulls the configuration of all the experiment IDs used by the application configuration through the API and stores the configuration in the file system, and after the application is started, periodically pulls a new experiment configuration from the API.
In the embodiment of the invention, when the application is started, the SDK of each language access pulls the configuration of all the experiment IDs used by the application configuration through the API and stores the configuration in a file system, if the application fails to be pulled and started, the experiment ID configuration error is exposed in time, after the application is started, new experiment configuration is periodically pulled from the API, and at the moment, the error log is recorded to keep the original configuration file unchanged if the pulling fails; the successful establishment of the experiment triggers an experiment updating event, a rule processor in the API triggers a rule reading compiler after receiving the event, and the API acquires the initial experiment configuration and compiles the initial experiment configuration into finished files in various formats respectively; the API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request; the API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, the subsequent SDK request needs to be provided with an ETag value on the subsequent SDK request, so that the ETag is kept consistent, the API considers that the configuration of the access end and the API end is consistent, and the network transmission data is reduced when the content is not responded.
When the initial downloading of all configured experiment IDs fails, the application is failed to start, so that problems are found and solved in time; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file; if the follow-up updating check is successful, the original configuration is covered; if the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
In step S103, according to the user access information, experimental calculation is performed, and the current scheme a or B is returned to the access application, and the scheme used by the user ID is recorded and stored.
In the embodiment of the invention, the developer using the experiment uses the access SDK of the corresponding language to access the access of the experiment. The SDK encapsulates the experiment configuration download function, the configuration parsing function and the calculation return response of the experiment to use a or B. The access personnel enter different logics according to the judgment of A or B; the user can run the access code when requesting, and the access code can analyze the experiment configuration file; transmitting a user identity ID, a user IP and a user session ID; judging whether the current access user is in a configured experimental area or not based on the IP of the user; if the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording; if the test area is in the test area, an experiment calculation stage is carried out, and an experiment log is recorded for subsequent analysis.
The key point of the experimental calculation is that if the length of hexadecimal representation of hash calculation is 1 bit, the value range is 0x 0-0 xF. If 4 bits are intercepted from the Hash value, the value range is 0x 0000-0 xFFFF, the Hash value/0 xFFFF is a value range of [0, 1], and the longer the Hash interception length is, the more dispersed the value is. If we divide the existing traffic into 100, experiments are done based on the user ID. The experiment ID is not changed, the user ID is not changed, the Hash value of the experiment ID and the user ID is not changed, and the result of taking the value as a fixed length value is not changed. If the Hash method adopts SHA1 for calculation, the experiment ID is 1, the user ID is 9678, the SHA1("1|9678") result is a hexadecimal representation "580b9483935ed29d66024789553bb8229f1d0506" with the length of 40, the value range of 40 bits is too wide, and the interception is performed to a certain extent, for example, only 20 bits are taken. To take a random number y of [1, 100], it can be rewritten as y ═ 1+ (100-1) × (x is a random number of 0 to 1), y ═ 1+ (100-1) × sha1("xxxx")/0 xffffn. This way it is calculated that the user will be fixed to an integer of 1 to 100. The initial stage of the experiment initialization uses fixed pattern AB weights configured by the experiment, if 40% uses A and 60% uses B, then 100 parts are initialized to 40 as A and 60 as B. And judging whether the integer randomly arrived by the user is on A or B, and responding to the caller to return A or B. The same strategy will be used for flow proportioning. And if the experiment entering the computing logic is processed in the issuing state, returning the option of the issuing result to the access party to process and execute the corresponding service logic.
And before the SDK responds to the calculation result, the experimental data is logged (including user id or session id, time of entering the experiment), and the log collection component is responsible for sending the recorded log data to the log analysis system.
In step S104, the recorded information is sent to the data analysis platform through the log collection system, and the analysis result and the experimental index data are displayed through the visualization platform.
In the embodiment of the invention, the log analysis system can analyze the experimental effect of each option AB by combining known data such as user access paths, page click data and the like, data processing is carried out in real time or offline, and the influence of each area AB option on each experimental index is displayed through an experimental effect visualization platform; and finally, after the AB scheme is confirmed, the experiment is issued in the background, and users who enter the experiment before now uniformly use the issued and used scheme execution logic.
Through the management background unified management experiment, the authority configuration is convenient for the cooperative management experiment of the personnel in the same department; developers only need to intervene in an experiment access stage, after the functions are on-line, the developers and operation and maintenance personnel do not need to intervene in operation, the operation can be completely carried out by products or operators, the experiment range is expanded, and finally, release and selection A or B do not need to be applied to release on-line; the experiment can be based on the user, and also can be based on the access session of the user, and the user always uses A or B under the same user or the same session, so that the consistency of experience is ensured. One application supports a plurality of experiments to be carried out in parallel, the same page supports a plurality of experiments to be carried out simultaneously, and the plurality of applications can share the same experiment to be carried out. The application range of the experiment can be controlled, the experiment can be performed only in a certain area, the influence surface is reduced, and the experiment can be stopped at any time to continue using the original logic. The computing logic is dispersed to each SDK for processing, and the dispersion of computing resources can not affect the management background or the API. The SDK access of a plurality of languages is supported, for example, PHP, node. js, Java and Python can all support the access test platform, so that the front-end and back-end operation and maintenance line products can be accessed and used for AB test.
Referring to fig. 2, fig. 2 provides an apparatus for managing an area AB experiment based on an algorithm, the apparatus comprising:
The experiment creating unit 21 is used for creating an experiment, receiving an initial experiment configuration and generating an experiment ID.
Optionally, the experiment creating unit 21 is further configured to obtain a login account ID of the user session or the user, configure an application area of the experiment, a user traffic ratio used by the experiment, and an AB option weight ratio.
And the acquired experiment configuration unit 22 is used for pulling the configuration of all the experiment IDs used by the application configuration through the API and storing the configuration in a file system when the application of each language access SDK is started, and periodically pulling new experiment configurations from the API after the application is started.
Optionally, the acquiring experiment configuration unit 22 is further configured to acquire the initial experiment configuration through an API, and compile the initial experiment configuration into configuration files in multiple formats respectively; the API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request; the API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, and the subsequent SDK request needs to be provided with an ETag value on the subsequent request of the SDK.
Optionally, the acquiring experiment configuration unit 22 is further configured to fail to start the application when the initial downloading of all configured experiment IDs fails; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file; if the follow-up updating check is successful, the original configuration is covered; if the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
The experiment calculation storage unit 23 is used for calculating an experiment according to the user access information, returning the experiment to the access application to use the scheme A or the scheme B currently, and recording and storing the scheme used by the user ID;
Optionally, the experiment calculation storage unit 23 is further configured to transmit a user identity ID, an IP of the user, and a session ID of the user; judging whether the current access user is in a configured experimental area or not based on the IP of the user; if the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording; if the test area is in the test area, entering an experimental calculation stage.
And the data analysis and display unit 24 is used for sending the recorded information to the data analysis platform through the log collection system and displaying the analysis structure and the experimental index data through the visualization platform.
In the embodiment of the present invention, each unit of the device for area AB experimental management based on algorithm implementation may be implemented by corresponding hardware or software unit, and each unit may be an independent software or hardware unit, or may be integrated into a software or hardware unit, which is not limited herein.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (6)
1. A method for managing an area AB experiment based on algorithm is characterized by comprising the following steps:
Creating an experiment, receiving initial experiment configuration and generating an experiment ID;
When the application is started, the SDK is accessed to each language, the configuration of all experiment IDs used by the application configuration is pulled through the API and stored in a file system, and after the application is started, new experiment configuration is periodically pulled from the API;
According to the user access information, performing experimental calculation, returning to the access application whether the scheme A or the scheme B is currently used, and recording and storing the scheme used by the user ID;
Sending the recorded information to a data analysis platform through a log collection system, and displaying an analysis result and experimental index data through a visual platform;
The initial experiment configuration comprises user session or login account ID of a user, application area of the configuration experiment, user flow ratio used by the experiment and AB option weight ratio;
Before the configuration of all experiment IDs used by the API pull application configuration is saved in the file system, the method further comprises the following steps:
The API acquires the initial experiment configuration and compiles the initial experiment configuration into configuration files in various formats respectively;
The API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request;
The API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, and the subsequent SDK request needs to be provided with an ETag value on the subsequent request of the SDK.
2. The method of claim 1, wherein the language access SDKs pull the latest configuration of all configured experiment IDs through the API at application startup and periodically pull new experiment configurations from the API, comprising:
When the initial downloading of all configured experiment IDs fails, the application is failed to start; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file;
If the follow-up updating check is successful, the original configuration is covered;
If the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
3. The method of claim 1, wherein the step of performing the calculation of the experiment based on the user access identity information comprises:
Transmitting a user identity ID, a user IP and a user session ID;
Judging whether the current access user is in a configured experimental area or not based on the IP of the user;
If the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording;
If the test area is in the test area, entering an experimental calculation stage.
4. An apparatus for managing an area AB experiment based on an algorithm, the apparatus comprising:
The experiment creating unit is used for creating an experiment, receiving initial experiment configuration and generating an experiment ID;
The method comprises the steps that an experiment configuration obtaining unit is used for pulling all the configuration of experiment IDs used by application configuration through an API and storing the configuration in a file system when the application of each language access SDK is started, and after the application is started, new experiment configuration is periodically pulled from the API;
The experiment calculation storage unit is used for calculating an experiment according to the user access information, returning the experiment to the access application to use the scheme A or the scheme B currently, and recording and storing the scheme used by the user ID;
The data analysis and display unit is used for sending the recorded information to the data analysis platform through the log collection system and displaying the analysis result and the experimental index data through the visual platform;
The initial experiment configuration comprises user session or login account ID of a user, application area of the configuration experiment, user flow ratio used by the experiment and AB option weight ratio;
The acquisition experiment configuration unit is also used for obtaining the initial experiment configuration by the API and compiling the initial experiment configuration into configuration files with various formats respectively; the API receives the initial experiment configuration reading request initiated by the SDK, configures a file with the corresponding experiment ID in a specified format, and responds to the SDK request; the API caches the latest updating time of the application ID, the application ID generates an ETag tag according to the updating time, the generated ETag tag is attached to response SDK request response information, and the subsequent SDK request needs to be provided with an ETag value on the subsequent request of the SDK.
5. The apparatus of claim 4, wherein the experiment configuration obtaining unit is further configured to, when the experiment IDs of all configurations downloaded for the first time fail, fail to start the application; detecting whether the configuration is updated or not regularly and continuously, and updating the configuration to a file; if the follow-up updating check is successful, the original configuration is covered; if the follow-up updating check fails, the original configuration is not covered, an error log is recorded, and a system maintenance worker is informed of checking response in time by using the function of the warning system, so that the access application system is not influenced.
6. The apparatus of claim 4, wherein the experiment calculation storage unit is further configured to transmit a user Identity (ID), a user IP, a user session ID; judging whether the current access user is in a configured experimental area or not based on the IP of the user; if the test area is not in the test area, returning to use the original default option, and walking the default logic without log recording; if the test area is in the test area, entering an experimental calculation stage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910294985.6A CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910294985.6A CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110008131A CN110008131A (en) | 2019-07-12 |
| CN110008131B true CN110008131B (en) | 2020-08-04 |
Family
ID=67171428
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910294985.6A Active CN110008131B (en) | 2019-04-12 | 2019-04-12 | Method and device for managing area AB experiment based on algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110008131B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110532192A (en) * | 2019-09-06 | 2019-12-03 | 北京温杜科技有限公司 | A kind of api interface performance optimized experimental facility |
| CN111708689B (en) * | 2020-05-19 | 2024-06-25 | 北京奇艺世纪科技有限公司 | Method and device for modifying AB experiment and electronic equipment |
| CN112817856A (en) * | 2021-02-04 | 2021-05-18 | 上海哔哩哔哩科技有限公司 | AB experiment integration method and system |
| CN113064820B (en) * | 2021-03-26 | 2022-09-16 | 深圳依时货拉拉科技有限公司 | Method, apparatus and computer readable storage medium for updating A/B experiment SDK |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020072873A1 (en) * | 2000-06-14 | 2002-06-13 | Liu Yi Bo | Method for managing and using test system |
| CN106856439A (en) * | 2016-12-07 | 2017-06-16 | 武汉斗鱼网络科技有限公司 | The method and server of a kind of scheme test |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577321B (en) * | 2012-08-07 | 2016-01-13 | 阿里巴巴集团控股有限公司 | A kind of multi version parallel test method of Website page and server |
| US20150012852A1 (en) * | 2013-07-08 | 2015-01-08 | Kobo Incorporated | User interface tool for planning an ab type of test |
| CN105718251B (en) * | 2014-12-03 | 2019-02-01 | 苏宁易购集团股份有限公司 | Multilingual SDK automates realization method and system |
| CN106341290A (en) * | 2016-08-31 | 2017-01-18 | 北京城市网邻信息技术有限公司 | Flow distribution server |
| CN107957938A (en) * | 2016-10-14 | 2018-04-24 | 北京酷我科技有限公司 | A kind of method and system for obtaining website test data |
| CN107451020B (en) * | 2017-06-28 | 2020-12-15 | 北京五八信息技术有限公司 | AB test system and test method |
| CN108345539A (en) * | 2018-01-31 | 2018-07-31 | 北京云测信息技术有限公司 | A kind of method and apparatus for carrying out AB tests |
-
2019
- 2019-04-12 CN CN201910294985.6A patent/CN110008131B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020072873A1 (en) * | 2000-06-14 | 2002-06-13 | Liu Yi Bo | Method for managing and using test system |
| CN106856439A (en) * | 2016-12-07 | 2017-06-16 | 武汉斗鱼网络科技有限公司 | The method and server of a kind of scheme test |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110008131A (en) | 2019-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110008131B (en) | Method and device for managing area AB experiment based on algorithm | |
| US11843661B2 (en) | Web service system and method | |
| KR100629057B1 (en) | Remote dynamic configuration of a web server to facilitate capacity on demand | |
| WO2009098909A1 (en) | Virtual appliance assignment system | |
| CN112329945A (en) | Model deployment and reasoning method and device | |
| CN113204730B (en) | Resource acquisition method, webvpn proxy server, system and server | |
| CN108848092A (en) | The processing method and processing device of micro services gray scale publication based on call chain | |
| CN107666525A (en) | The method and apparatus of cluster container IP distribution | |
| CN111708702A (en) | Simulation test method, client, server, system and readable storage medium | |
| CN105262608B (en) | Monitoring method and device for network service | |
| US20150195128A1 (en) | Apparatus and method for supporting configuration management of virtual machine, and apparatus and method for brokering cloud service using the configuration management supporting apparatus | |
| CN106897609A (en) | The method and device that a kind of application program to dynamic load is monitored | |
| CN108737463A (en) | A kind of software deployment method, server and system | |
| CN109783151B (en) | Method and device for rule change | |
| EP3616061B1 (en) | Hyper dynamic java management extension | |
| EP2808792B1 (en) | Method and system for using arbitrary computing devices for distributed data processing | |
| CN109783197A (en) | Dispatching method and device for program runtime environment | |
| CN108076125A (en) | Interface allocation method and system | |
| EP3079339A1 (en) | Method, device, and esb system for data processing | |
| US20090257085A1 (en) | Generation of a web page including menu items for web pages | |
| CN111831503B (en) | Monitoring method based on monitoring agent and monitoring agent device | |
| CN113778499B (en) | Method, apparatus, device and computer readable medium for publishing services | |
| CN113132447A (en) | Reverse proxy method and system | |
| CN113238886A (en) | Method and system for testing functional service and terminal equipment | |
| CN111338647B (en) | Big data cluster management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |