CN112446574B - Product evaluation method, device, electronic equipment and storage medium - Google Patents
Product evaluation method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112446574B CN112446574B CN201910819016.8A CN201910819016A CN112446574B CN 112446574 B CN112446574 B CN 112446574B CN 201910819016 A CN201910819016 A CN 201910819016A CN 112446574 B CN112446574 B CN 112446574B
- Authority
- CN
- China
- Prior art keywords
- group data
- data
- experimental
- users
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 109
- 238000003860 storage Methods 0.000 title claims abstract description 27
- 238000002474 experimental method Methods 0.000 claims description 92
- 238000000034 method Methods 0.000 claims description 43
- 238000012360 testing method Methods 0.000 claims description 29
- 230000015654 memory Effects 0.000 claims description 19
- 238000013179 statistical model Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 12
- 238000004458 analytical method Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
Landscapes
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Engineering & Computer Science (AREA)
- Strategic Management (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Educational Administration (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a product evaluation method, a device, electronic equipment and a storage medium, and relates to the technical field of big data. The specific implementation scheme is that aiming at each time period of M time periods, user data of N experimental groups and user data of N control groups are respectively obtained, M is an integer greater than or equal to 1, and N is an integer greater than 1; summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; summarizing the user data of each control group respectively to obtain N pieces of first control group data; according to N first experimental group data and N first comparison group data which correspond to the M time periods respectively, evaluating an experimental product and a comparison product to obtain an evaluation result; the evaluation result is pushed to the user, the product evaluation is realized, and a large amount of computing resources are saved.
Description
Technical Field
The application relates to the computer technology, in particular to the technical field of big data.
Background
In the products of the internet, since the iteration of the product is very rapid, in order to ensure the correct iteration direction of the product, each product iteration needs to be evaluated, so as to judge whether the experience of the product iteration to the user is optimized or not and whether the product affects the business income of the company or not.
In the prior art, in order to realize the evaluation of product iteration, an online control small flow experiment system (an AB Test small flow evaluation system) is generally used for testing, and the AB Test is an effective and refined operation means. And in the same time dimension, visitor groups with the same (or similar) composition components are respectively made to randomly access products of different versions, user experience data and business data of each group are collected, and finally, the best version is analyzed and evaluated for formal adoption.
However, in the prior art, the method of evaluating the product occupies a large amount of computing resources.
Disclosure of Invention
The embodiment of the application provides a product evaluation method, a device, electronic equipment and a storage medium, which realize product evaluation and save a large amount of computing resources.
In a first aspect, an embodiment of the present application provides a product evaluation method, including:
for each of the M time periods, respectively acquiring user data of N experimental groups and user data of N control groups, wherein the user data of each experimental group is user data obtained by at least one user using an experimental product, the user data of each control group is user data obtained by at least one user using a control product, M is an integer greater than or equal to 1, and N is an integer greater than 1; summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; summarizing the user data of each control group respectively to obtain N pieces of first control group data; according to N first experimental group data and N first comparison group data which correspond to the M time periods respectively, evaluating an experimental product and a comparison product to obtain an evaluation result; pushing the evaluation result to the user.
In this embodiment of the present application, by respectively acquiring, for each time period on M time periods, user data of N experiment groups and user data of N control groups, and respectively summarizing the user data of each experiment group in the N experiment groups, N first experiment group data are obtained, and respectively summarizing the user data of each control group in the N control groups, N first control group data are obtained, and then according to the N first experiment group data and the N first control group data, a comparison analysis is performed on experiment products and control products, which can save a large amount of computing resources.
In one implementation manner, the evaluating the experimental product and the control product according to the N first experimental group data and the N first control group data corresponding to the M time periods respectively to obtain an evaluation result includes:
respectively determining N groups of first experimental group data to be summarized and N groups of first comparison group data to be summarized, wherein each group of first experimental group data to be summarized comprises one first experimental group data on each of M time periods, and each group of first comparison group data to be summarized comprises one first comparison group data on each of M time periods; respectively summarizing N groups of first experimental group data to be summarized to obtain N second experimental group data, and respectively summarizing N groups of first control group data to be summarized to obtain N second control group data; and evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain an evaluation result.
According to the embodiment of the application, the first experimental group data of each experimental group in M time periods are summarized, and the first control group data of each control group in M time periods are summarized, so that the comparison analysis of experimental products and control products in M time periods can be realized, the experimental products and the control products in different time periods can be evaluated, and the calculation resources are further saved.
In one implementation manner, before acquiring the user data of the N experimental groups and the user data of the N control groups respectively, the method further includes:
acquiring respective identifications of a plurality of first users and respective identifications of a plurality of second users; dividing the plurality of first users into N experiment groups according to the respective identifications of the plurality of first users, and dividing the plurality of second users into N experiment groups according to the respective identifications of the plurality of second users.
In the embodiment of the application, the plurality of first users are divided into the N experiment groups through the respective identifications of the plurality of first users, and the plurality of second users are divided into the N comparison groups according to the respective identifications of the plurality of second users, so that the experiment groups or the comparison groups where the user data of the same user are located are ensured to be determined, the independence of the first experiment group data and the first comparison group data after the user data in the same experiment group or the comparison group are collected later is ensured, and the reliability of the confidence degree of evaluating the evaluating experiment products and the comparison products is further ensured.
In one implementation, the dividing the plurality of first users into N experiment groups according to the respective identities of the plurality of first users includes:
determining N first hash values corresponding to the identifiers of the first users; the plurality of first users are divided into N experiment groups according to the N first hash values.
In the embodiment of the application, the stability of the experiment group where each first user is located is effectively ensured through the hash algorithm.
In one implementation, the dividing the plurality of second users into N experiment groups according to the respective identities of the plurality of second users includes:
determining N second hash values corresponding to the identifiers of the second users respectively; and dividing the plurality of second users into N comparison groups according to the N second hash values.
In the embodiment of the application, the stability of the control group where each second user is located is effectively ensured through the hash algorithm.
In one implementation, the evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain the evaluation result includes:
selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer which is more than or equal to 1 and less than or equal to N; and inputting the P second experimental group data and the P second control group data into a statistical model of the double-population T test to obtain an evaluation result.
In the embodiment of the application, the experimental product and the comparison product are evaluated by arbitrarily selecting a plurality of second experimental group data and arbitrarily selecting a plurality of second comparison group data to be input into the statistical model of the double-overall T test, and the reliability of the evaluation result is improved.
In one implementation, the evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain an evaluation result includes:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data; randomly selecting N-1 different second control group data from the N second control group data for summarizing until each second control group data is selected for N-1 times to obtain N third control group data; n third experimental group data and N third control group data are input into a statistical model of the double-population T test to obtain an evaluation result.
In the embodiment of the application, the third control group data and the third experimental group data are obtained in the above manner, and the N third experimental group data and the N third control group data are input into the statistical model of the double-population T test, so that the accuracy of evaluating the experimental product and the control product is improved.
Optionally, the product evaluation method provided in the embodiment of the present application may further include:
n first experimental group data and N first control group data were stored.
In the embodiment of the present application, N pieces of first experiment group data are data that are summarized by user data of each experiment group; the N first comparison group data are data obtained by summarizing the user data of each comparison group respectively, and the N first experiment group data and the N first comparison group data are stored, so that the storage space is saved.
The following describes an apparatus, an electronic device, a computer readable storage medium, and a computer program product provided in the embodiments of the present application, and the content and effects thereof may refer to a product evaluation method provided in the embodiments of the present application, which are not described in detail.
In a second aspect, embodiments of the present application provide a product assessment device, including:
the first acquisition module is used for respectively acquiring user data of N experimental groups and user data of N comparison groups according to each time period of M time periods, wherein the user data of each experimental group is user data obtained by at least one user using an experimental product, the user data of each comparison group is user data obtained by at least one user using a comparison product, M is an integer greater than or equal to 1, and N is an integer greater than 1; the processing module is used for summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; summarizing the user data of each control group respectively to obtain N pieces of first control group data; the evaluation module is used for evaluating the experimental product and the control product according to N first experimental group data and N first control group data which correspond to the M time periods respectively so as to obtain an evaluation result; and the pushing module is used for pushing the evaluation result to the user.
In one implementation, the evaluation module is specifically configured to:
respectively determining N groups of first experimental group data to be summarized and N groups of first comparison group data to be summarized, wherein each group of first experimental group data to be summarized comprises one first experimental group data on each of M time periods, and each group of first comparison group data to be summarized comprises one first comparison group data on each of M time periods; respectively summarizing N groups of first experimental group data to be summarized to obtain N second experimental group data, and respectively summarizing N groups of first control group data to be summarized to obtain N second control group data; and evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain an evaluation result.
Optionally, the product evaluation device provided in the embodiment of the present application further includes:
the second acquisition module is used for acquiring the respective identifications of the plurality of first users and the respective identifications of the plurality of second users; the dividing module is used for dividing the plurality of first users into N experiment groups according to the respective identifications of the plurality of first users and dividing the plurality of second users into N experiment groups according to the respective identifications of the plurality of second users.
Optionally, the dividing module is specifically configured to:
determining N first hash values corresponding to the identifiers of the first users; the plurality of first users are divided into N experiment groups according to the N first hash values.
Optionally, the partitioning module is further configured to:
determining N second hash values corresponding to the identifiers of the second users respectively; and dividing the plurality of second users into N comparison groups according to the N second hash values.
Optionally, the evaluation module is further configured to:
selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer which is more than or equal to 1 and less than or equal to N; and inputting the P second experimental group data and the P second control group data into a statistical model of the double-population T test to obtain an evaluation result.
Optionally, the evaluation module is further configured to:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data; randomly selecting N-1 different second control group data from the N second control group data for summarizing until each second control group data is selected for N-1 times to obtain N third control group data; n third experimental group data and N third control group data are input into a statistical model of the double-population T test to obtain an evaluation result.
Optionally, the product evaluation device provided in the embodiment of the present application further includes:
the storage module is used for storing N first experiment group data and N first control group data.
In a third aspect, an embodiment of the present application provides an electronic device, including:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method as provided by the first aspect or an implementation of the first aspect.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method as provided by the first aspect or an implementation of the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising: executable instructions for implementing a method as provided in the first aspect or the realizations of the first aspect.
One embodiment of the above application has the following advantages or benefits: the method comprises the steps of respectively acquiring user data of N experimental groups and user data of N control groups according to each time period of M time periods, respectively summarizing the user data of each experimental group in the N experimental groups to obtain N first experimental group data, respectively summarizing the user data of each control group in the N control groups to obtain N first control group data, and then carrying out comparative analysis on experimental products and control products according to the N first experimental group data and the N first control group data, so that a large amount of calculation resources can be saved.
Other effects of the above alternative will be described below in connection with specific embodiments.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is an exemplary application scenario diagram provided by an embodiment of the present application;
FIGS. 2A-2B are schematic diagrams of a user's terminal interface provided in an embodiment of the present application;
FIG. 3 is a flow chart of a method for evaluating a product according to an embodiment of the present application;
FIG. 4 is a flow chart of a method for product assessment according to another embodiment of the present application;
FIG. 5 is a schematic diagram of a first user grouping manner provided in an embodiment of the present application;
fig. 6 is a schematic diagram of a second user grouping manner provided in an embodiment of the present application;
FIG. 7 is a flow chart of a method for product assessment according to yet another embodiment of the present application;
FIG. 8 is a schematic diagram of a product evaluation device according to an embodiment of the present disclosure;
FIG. 9 is a schematic view of a product assessment device according to another embodiment of the present application;
fig. 10 is a block diagram of an electronic device of a product assessment method according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the products of the internet, since the iteration of the product is very rapid, in order to ensure the correct iteration direction of the product, each product iteration needs to be evaluated, so as to judge whether the experience of the product iteration to the user is optimized or not and whether the product affects the business income of the company or not. To achieve the evaluation of product iterations, testing is typically performed using an AB Test low flow evaluation system. AB testing is the creation of two (A/B) or more (A/B/n) versions for a Web (Web page) or App (application) interface or flow. And in the same time dimension, visitor groups with the same (or similar) composition components are respectively made to randomly access products of different versions, user experience data and business data of each group are collected, and finally, the best version is analyzed and evaluated for formal adoption. For example, when the AB test is performed on the product, two schemes can be formulated for the same optimization target, so that one part of users use the A scheme, and the other part of users use the B scheme, and indexes such as conversion rate, click rate, retention rate and the like of different schemes are counted and compared to judge the advantages and disadvantages of the different schemes and make decisions, and the conversion rate is improved. However, in the prior art, the method of evaluating the product occupies a large amount of storage and computing resources. In order to solve the technical problems, embodiments of the present application provide a product evaluation method, a device, an electronic apparatus, and a storage medium.
In the following, an exemplary application scenario of the embodiments of the present application is described.
The product evaluation method provided in the embodiment of the present application may be executed by the product evaluation device provided in the embodiment of the present application, where the product evaluation device provided in the embodiment of the present application may be part or all of a terminal device, and fig. 1 is an exemplary application scenario diagram provided in the embodiment of the present application, as shown in fig. 1, where the product evaluation method provided in the embodiment of the present application may be applied to a terminal device 11, where data communication exists between the terminal device 11 and a server 12, and the terminal device 11 may obtain, through the server 12, user data of N experimental groups and user data of N comparison groups. The product evaluation may be, for example, an evaluation of an updated version of a Web or App interface or a process production version, to determine whether the experimental product is more convenient for a user to use than a control product, etc., and the product type of the product evaluation in the embodiment of the present application is not limited.
In a possible implementation, taking a product as an Application (APP) as an example, fig. 2A-2B are schematic views of a terminal interface of a user provided in this embodiment of the present application, where an existing application version is application 1.0, with iterative updating of the product, a technician now develops application 2.0, and in the same time dimension, multiple visitor groups with the same (or similar) composition use application 1.0 and application 2.0 respectively. As shown in fig. 2A, the interface 21 is a terminal interface of a user using the application 1.0, and as shown in fig. 2B, the interface 22 is a terminal interface of a user using the application 2.0, and the type of a terminal used by the user is not limited in the embodiment of the present application, and may be, for example, a mobile phone, a personal computer, a tablet computer, a wearable device, a vehicle-mounted terminal, and the like. The server collects user experience data and business data of each group and sends the user experience data and business data to the terminal equipment, and the terminal equipment analyzes and evaluates the best version formally adopted.
Based on this, the embodiment of the application provides a product evaluation method, a device, an electronic device and a storage medium, and the embodiment of the application is described below.
Fig. 3 is a flow chart of a method for evaluating a product according to an embodiment of the present application, where the method may be performed by a device for evaluating a product, and the device may be implemented by software and/or hardware, for example: the device may be a client or a terminal device, where the terminal device may be a personal computer, a smart phone, a user terminal, a tablet computer, a wearable device, etc., and the product evaluation method is described below with the terminal device as an execution body, as shown in fig. 3, where the product evaluation method provided in the embodiment of the present application may include:
step S101: for each of the M time periods, the user data of N experimental groups and the user data of N control groups are acquired, respectively.
The user data of each experiment group is user data obtained by at least one user using an experiment product, the number of the experiment groups and the number of the users in each experiment group are not limited, and the user data of each experiment group can comprise flow consumed by the user when the user uses the experiment product and/or click rate of the user on the experiment product; the user data of each control group is user data obtained by at least one user using the control product, and the number of the control groups and the number of users in each control group are not limited in the embodiment of the present application, and the user data of each control group may include flow consumed by the user when using the control product and/or click rate of the user on the control product.
For each of M time periods, the length of one time period is not limited, for example, one time period is one day, and in at least one day, the user data of N experimental groups in each day and the user data of N control groups in each day are respectively obtained. For another example, if one time period is one week, the user data of N experimental groups in each week and the user data of N control groups in each week are respectively obtained. In addition, the embodiment of the present application also does not limit the number M of acquired time periods, M being an integer greater than or equal to 1.
The user data of each experimental group and the user data of each control group may be the same in type, and the user data of each experimental group may include a flow rate consumed by a user when using an experimental product, a click rate of the user on the experimental product, and a flow rate consumed by the user when using the experimental product and a click rate of the user on the experimental product; the user data for each control group may include a flow rate consumed by the user when using the control product, a click rate of the control product by the user, and a flow rate consumed by the user when using the control product and a click rate of the control product by the user; the data types of the user data are not limited to this, and for different experimental products and comparison products, the required analysis data may also be different, so that the user may set according to the requirement to obtain the data types required by the user, for example, the user data may further include consumption data, motion data, playing times, and the like, which is not limited in this embodiment of the present application.
Step S102: summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; and summarizing the user data of each control group respectively to obtain N pieces of data of the first control group.
Taking each time period as an example of one day, the day at this time may be 0 to 24 points, or may be 12 to 12 points on the next day, as long as the continuous 24 hours are satisfied, which is not limited in the embodiment of the present application. And summarizing the user data of each experimental group in the N experimental groups respectively in the user data of the N experimental groups acquired in one day, so that N pieces of first experimental group data can be obtained. The user data of each experiment group may be summarized, for example, the user data of the same category may be summarized, for example, the flow consumed when the user uses the experiment product in each experiment group may be summarized, and the click rate of the user on the experiment product may be summarized. The manner in which the user data of each experimental group is summarized in the embodiment of the present application is not limited. The user data of each of the N control groups is summarized respectively to obtain N first control group data, and similarly to the above manner, specifically, the user data of each of the N control groups is summarized respectively from the user data of the N control groups obtained in one day, so that N first control group data can be obtained. The user data of each control group may be summarized, and the user data may be summarized by the same type, for example, by respectively summarizing the flow consumed when the user uses the control product in each control group, and by respectively summarizing the click rate of the user on the control product. The method for summarizing the user data of each control group is not limited in the embodiment of the present application.
After obtaining the N first experimental group data and the N first control group data, in a possible implementation manner, the product evaluation method provided by the embodiment of the application may further include:
n first experimental group data and N first control group data were stored.
The embodiment of the application does not limit the locations where the N first experimental group data and the N first control group data are stored.
Step S103: and evaluating the experimental product and the control product according to N first experimental group data and N first control group data which correspond to the M time periods respectively so as to obtain an evaluation result.
After the N first experimental group data and the N first control group data are obtained, comparing and analyzing the experimental product and the control product according to the N first experimental group data and the N first control group data respectively obtained in the M time periods so as to evaluate the experimental product and the control product and further obtain an evaluation result. In a possible implementation manner, the evaluation of the experimental product and the control product may be implemented by a manner of pre-establishing a neural network model, for example, establishing a relationship between the N first experimental group data, the N first control group data, and judgment indexes of the experimental product and the control product, where the judgment indexes may be indexes for judging the optimization degree of the experimental product and the control product, for example: user usage, conversion, retention, etc. The embodiments of the present application are not limited in this regard.
Step S104, pushing the evaluation result to the user.
By evaluating the experimental product and the control product, an evaluation result can be obtained, and in order to facilitate the analysis of the evaluation result by a user, the evaluation result can be pushed to the user. The embodiment of the present application does not limit the specific form of pushing the evaluation result to the user, for example, the evaluation result may be pushed in forms of a table, a graph, a text, etc., and the embodiment of the present application is not limited thereto.
In the embodiment of the application, the user data of the N experimental groups and the user data of the N control groups are respectively acquired in M time periods, the user data of each experimental group in the N experimental groups is summarized, the user data of each control group in the N control groups is summarized, a large amount of storage space can be saved, and then the comparison analysis is performed on the experimental product and the control product according to the N first experimental group data and the N first control group data, so that a large amount of calculation resources can be saved.
In order to ensure independence of the first experimental group data and the first control group data, and further improve reliability of evaluating confidence of the experimental product and the control product evaluation, in a possible implementation, fig. 4 is a schematic flow chart of a product evaluation method provided in another embodiment of the present application, where the method may be performed by a product evaluation device, and the device may be implemented by software and/or hardware, for example: the device may be a client or a terminal device, where the terminal device may be a personal computer, a smart phone, a user terminal, a tablet pc, a wearable device, etc., and the product evaluation method is described below with the terminal device as an execution body, as shown in fig. 4, before step S101, the product evaluation method provided in the embodiment of the present application further includes:
Step S201: the method comprises the steps of obtaining respective identifications of a plurality of first users and respective identifications of a plurality of second users.
The first user uses the experimental product, the second user uses the comparison product, the first user needs to be determined before the user data of the N experimental groups are acquired, the second user needs to be determined before the user data of the N comparison groups are acquired, and the mode, the number and the like for determining the first user and the second user are not limited in the embodiment of the application. For example, if the primary user of the experimental product and the control product is a college student, a part of the students in a certain college may be determined as a first user, another part of the students in the college may be determined as a second user, or a first user may be determined as a college student of a certain college, and a second user may be determined as a college student of another same level college.
Whether the first user or the second user exists, each user has a corresponding identifier, for example, the identifier can be information such as an ID (identity) of a mobile phone of the user, a device number and the like, and the type of the identifier of each user is not limited in the embodiment of the application. The embodiment of the application does not limit the specific implementation manner of acquiring the respective identifications of the plurality of first users and the respective identifications of the plurality of second users.
Step S202: dividing the plurality of first users into N experiment groups according to the respective identifications of the plurality of first users, and dividing the plurality of second users into N experiment groups according to the respective identifications of the plurality of second users.
The embodiment of the application does not limit the specific implementation manner that the plurality of first users can be divided into N experiment groups according to the respective identifiers of the first users, for example, the embodiment can be implemented by randomly establishing experiment group lists, and by way of example, N experiment group lists are randomly established, each experiment group list includes the identifier of the experiment group, for example, what experiment group is the number of the experiment group, and the identifier of the first user, and each first user can only be in one experiment group list. The embodiments of the present application are not limited in this regard. The method for dividing the plurality of second users into N comparison groups according to the respective identifications of the plurality of second users is similar to the method for dividing the plurality of first users into N experiment groups according to the respective identifications of the plurality of first users, and will not be described again here.
In order to divide the plurality of first users into N experiment groups according to the respective identifiers of the plurality of first users, in one possible implementation, dividing the plurality of first users into N experiment groups according to the respective identifiers of the plurality of first users may include:
Determining N first hash values corresponding to the identifiers of the first users; the plurality of first users are divided into N experiment groups according to the N first hash values.
N first hash values corresponding to the respective identifications of the plurality of first users can be obtained through the identifications of the plurality of first users and the hash algorithm, and the embodiment of the application does not limit the specific algorithm of the hash algorithm. After determining the N first hash values, dividing the plurality of first users into N experiment groups according to the N first hash values, where in a possible implementation manner, the first hash values may be implemented by setting first correspondence relationships of the N first hash values and the identifiers of the N experiment groups in a one-to-one correspondence manner, where the first correspondence relationships may be implemented by a first correspondence relationship list, which is not limited in this embodiment, and in addition, a specific form of the first hash values is not limited in this embodiment. And determining the experiment group where each first user is located through the first hash value and the first corresponding relation. Fig. 5 is a schematic diagram of a first user grouping manner provided in the embodiment of the present application, as shown in fig. 5, where a plurality of first users include respective identifiers of a plurality of first users, N hash values may be obtained through a hash algorithm, N is equal to 3 in fig. 5 as an example, and the embodiment of the present application is not limited thereto, and the 3 hash values respectively correspond to a first experiment group, a second experiment group and a third experiment group, where the number of users in each experiment group is not limited, and finally the plurality of first users are divided into three different experiment groups, and the users in each experiment group are fixed users.
In the embodiment of the application, the stability of the experiment group where each first user is located is effectively ensured through the hash algorithm.
In order to achieve the division of the plurality of second users into N experimental groups according to their respective identities, in one possible implementation, the division of the plurality of second users into N experimental groups according to their respective identities comprises:
determining N second hash values corresponding to the identifiers of the second users respectively; and dividing the plurality of second users into N comparison groups according to the N second hash values.
N second hash values corresponding to the respective identifications of the plurality of second users can be obtained through the identifications of the plurality of second users and the hash algorithm, and the embodiment of the invention does not limit the specific algorithm of the hash algorithm. In a possible implementation manner, the dividing the plurality of second users into the N comparison groups according to the N second hash values may be implemented by setting a second correspondence between the N second hash values and the identifiers of the N comparison groups, where the second correspondence may be implemented by a second correspondence list, which is not limited by the embodiment of the present application, and in addition, a specific form of the second hash value is not limited by the embodiment of the present application. And determining the control group where each second user is located through the second hash value and the second corresponding relation. Fig. 6 is a schematic diagram of a second user grouping manner provided in the embodiment of the present application, as shown in fig. 6, where a plurality of second users include respective identifiers of a plurality of second users, N hash values may be obtained by a hash algorithm, N is equal to 3 in fig. 6, which is taken as an example, and the embodiment of the present application is not limited thereto, and the 3 hash values respectively correspond to a first comparison group, a second comparison group, and a third comparison group, where the number of users in each comparison group is not limited, and finally the plurality of second users are divided into three different comparison groups, and the users in each comparison group are fixed users.
In the embodiment of the application, the stability of the control group where each second user is located is effectively ensured through the hash algorithm.
In the embodiment of the application, the plurality of first users are divided into the N experiment groups through the respective identifications of the plurality of first users, and the plurality of second users are divided into the N comparison groups according to the respective identifications of the plurality of second users, so that the experiment groups or the comparison groups where the user data of the same user are located are ensured to be determined, the independence of the first experiment group data and the first comparison group data after the user data in the same experiment group or the comparison group are collected later is ensured, and the reliability of the confidence degree of evaluating the evaluating experiment products and the comparison products is further ensured.
In order to evaluate the experimental product and the control product according to the N first experimental group data and the N first control group data corresponding to the M time periods respectively to obtain the evaluation result, in one possible manner, fig. 7 is a schematic flow chart of a product evaluation method provided in a further embodiment of the present application, where the method may be executed by a product evaluation device, and the device may be implemented by software and/or hardware, for example: the device may be a client or a terminal device, where the terminal device may be a personal computer, a smart phone, a user terminal, a tablet pc, a wearable device, etc., and the product evaluation method is described below with the terminal device as an execution body, as shown in fig. 7, step S103 in the product evaluation method provided in the embodiment of the present application may include:
Step S301: and respectively determining N groups of first experimental group data to be summarized and N groups of first control group data to be summarized.
Each set of first experimental group data to be summarized comprises one first experimental group data over each of the M time periods, and each set of first control group data to be summarized comprises one first control group data over each of the M time periods.
Step S302: and respectively summarizing N groups of first experimental group data to be summarized to obtain N second experimental group data, and respectively summarizing N groups of first control group data to be summarized to obtain N second control group data.
The second experimental group data corresponding to each experimental group is summarized data of the first experimental group data of the experimental group in M time periods, the second control group data corresponding to each experimental group is summarized data of the first control group data of the control group in M time periods, and the specific time periods of the M time periods are not limited, and the length of the time periods is not limited.
Step S303: and evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain an evaluation result.
According to the N second experimental group data and the N second control group data, the experimental products and the control products are evaluated to obtain the implementation manner of the evaluation result, which can be referred to the description in step S103 in the embodiment of the present application and will not be described in detail. The embodiments of the present application are not limited in this regard.
In one possible embodiment, the evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain the evaluation result includes:
selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer greater than or equal to 1 and less than or equal to N; and inputting the P second experimental group data and the P second control group data into a statistical model of the double-population T test to obtain an evaluation result.
According to the embodiment of the application, the size of P is not limited, P pieces of second experiment group data are selected randomly from the second experiment group data corresponding to N experiment groups, P pieces of second comparison group data are selected from the second comparison group data corresponding to N comparison groups, the selected P pieces of second experiment group data and P pieces of second comparison group data are input into a statistical model of double-overall T test, whether the average number of two samples is obvious from the overall represented by the average number of the two samples is checked, so that the evaluation of experimental products and comparison products is realized, and the reliability of the evaluation result is improved.
In another possible embodiment, the evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain the evaluation result includes:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data; randomly selecting N-1 different second control group data from the N second control group data for summarizing until each second control group data is selected for N-1 times to obtain N third control group data; n third experimental group data and N third control group data are input into a statistical model of the double-population T test to obtain an evaluation result.
N-1 different second experimental group data are selected randomly from the second experimental group data corresponding to the N experimental groups to be summarized until each second experimental group data is selected N-1 times, N third experimental group data are obtained, reliability of the third experimental group data is improved, N-1 different second comparison group data are selected randomly from the second experimental group data corresponding to the N experimental groups to be summarized until each second comparison group data is selected N-1 times, N third comparison group data are obtained, reliability of the third comparison group data is improved, and the N third experimental group data and the N third comparison group data are input into a statistical model of double overall T test, so that accuracy of evaluating experimental products and comparison products is improved.
According to the embodiment of the application, the first experimental group data of each experimental group in M time periods are summarized, and the first control group data of each control group in M time periods are summarized, so that the comparison analysis of experimental products and control products in M time periods can be realized, the experimental products and the control products in different time periods are evaluated, and the calculation resources are further saved.
The product evaluation device, the electronic device, and the readable storage medium provided in the embodiments of the present application are described below, and the content and effects thereof may refer to the above method embodiments and are not described in detail.
An embodiment of the present application provides a product analysis device, and fig. 8 is a schematic structural diagram of a product evaluation device provided in an embodiment of the present application, where the device may be implemented in a software and/or hardware manner, for example: the device may be a client or a terminal device, and the terminal device may be a personal computer, a smart phone, a user terminal, a tablet computer, a wearable device, etc., as described in fig. 8, a product evaluation device provided in an embodiment of the present application may include:
a first obtaining module 81, configured to obtain, for each of M time periods, user data of N experimental groups and user data of N control groups, where the user data of each experimental group is user data obtained by at least one user using an experimental product, the user data of each control group is user data obtained by at least one user using a control product, M is an integer greater than or equal to 1, and N is an integer greater than 1;
The processing module 82 is configured to summarize, for each time period, the user data of each experimental group, to obtain N pieces of data of the first experimental group; summarizing the user data of each control group respectively to obtain N pieces of first control group data;
the evaluation module 83 is configured to evaluate the experimental product and the control product according to N pieces of first experimental group data and N pieces of first control group data corresponding to the M time periods, respectively, so as to obtain an evaluation result;
a pushing module 84, configured to push the evaluation result to the user.
Optionally, the evaluation module 83 is specifically configured to:
respectively determining N groups of first experimental group data to be summarized and N groups of first comparison group data to be summarized, wherein each group of first experimental group data to be summarized comprises one first experimental group data on each of M time periods, and each group of first comparison group data to be summarized comprises one first comparison group data on each of M time periods; respectively summarizing N groups of first experimental group data to be summarized to obtain N second experimental group data, and respectively summarizing N groups of first control group data to be summarized to obtain N second control group data; and evaluating the experimental product and the control product according to the N second experimental group data and the N second control group data to obtain an evaluation result.
Optionally, the evaluation module 83 is further configured to:
selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer which is more than or equal to 1 and less than or equal to N; and inputting the P second experimental group data and the P second control group data into a statistical model of the double-population T test to obtain an evaluation result.
Optionally, the evaluation module 83 is further configured to:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data; randomly selecting N-1 different second control group data from the N second control group data for summarizing until each second control group data is selected for N-1 times to obtain N third control group data; n third experimental group data and N third control group data are input into a statistical model of the double-population T test to obtain an evaluation result.
In one implementation manner, fig. 9 is a schematic structural diagram of a product evaluation device according to another embodiment of the present application, where the device may be implemented in a software and/or hardware manner, for example: the device may be a client or a terminal device, where the terminal device may be a personal computer, a smart phone, a user terminal, a tablet computer, a wearable device, etc., as described in fig. 9, a product evaluation device provided in an embodiment of the present application may further include:
A second obtaining module 85, configured to obtain respective identifiers of the plurality of first users and respective identifiers of the plurality of second users;
the dividing module 86 is configured to divide the plurality of first users into N experiment groups according to the respective identifiers of the plurality of first users, and divide the plurality of second users into N experiment groups according to the respective identifiers of the plurality of second users.
Optionally, the dividing module 86 is specifically configured to:
determining N first hash values corresponding to the identifiers of the first users; the plurality of first users are divided into N experiment groups according to the N first hash values.
Optionally, the partitioning module 86 is further configured to:
determining N second hash values corresponding to the identifiers of the second users respectively; and dividing the plurality of second users into N comparison groups according to the N second hash values.
Optionally, as shown in fig. 9, the product evaluation method provided in the embodiment of the present application may further include:
the storage module 87 is configured to store N first experimental group data and N first control group data.
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
Fig. 10 is a block diagram of an electronic device of a product assessment method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 10, the electronic device includes: one or more processors 901, memory 902, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). In fig. 10, a processor 901 is taken as an example.
Memory 902 is a non-transitory computer-readable storage medium provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the product assessment method provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the product assessment method provided by the present application.
The memory 902 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the acquisition module 81, the processing module 82, the evaluation module 83, the determination module 84, and the grouping module 85 shown in fig. 9) corresponding to the product evaluation method in the embodiments of the present application. The processor 901 performs various functional applications of the server and data processing, i.e., implements the product evaluation method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 902.
The memory 902 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created according to the use of the electronic device of the product evaluation method, and the like. In addition, the memory 902 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 902 optionally includes memory remotely located relative to processor 901, which may be connected to the electronic device of the product assessment method via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the product evaluation method may further include: an input device 903 and an output device 904. The processor 901, memory 902, input devices 903, and output devices 904 may be connected by a bus or other means, for example in fig. 10.
The input device 903 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device of the product assessment method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. input devices. The output means 904 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the user data of the N experimental groups and the user data of the N control groups are respectively acquired in at least one time period, the user data of each experimental group in the N experimental groups are respectively summarized, the user data of each control group in the N control groups are respectively summarized, and then the comparison analysis is carried out on experimental products and control products according to the N first experimental group data and the N first control group data, so that the technical problem that a method for evaluating products in the prior art can occupy a large amount of storage and calculation resources is solved, and the technical effects of saving a large amount of storage space and saving a large amount of calculation resources are achieved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (17)
1. A method of product assessment, comprising:
for each of M time periods, respectively acquiring user data of N experimental groups and user data of N comparison groups, wherein the user data of each experimental group is user data obtained by at least one user using an experimental product, the user data of each comparison group is user data obtained by at least one user using a comparison product, M is an integer greater than 1, and N is an integer greater than 1;
Summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; summarizing the user data of each control group respectively to obtain N pieces of first control group data;
respectively determining N groups of to-be-summarized first experimental group data and N groups of to-be-summarized first control group data, wherein each group of to-be-summarized first experimental group data comprises one first experimental group data on each of the M time periods, and each group of to-be-summarized first control group data comprises one first control group data on each of the M time periods;
summarizing N groups of data of the first experimental group to be summarized respectively to obtain N second experimental group data, and summarizing N groups of data of the first comparison group to be summarized respectively to obtain N second comparison group data;
evaluating the experimental products and the control products according to the N pieces of second experimental group data, the N pieces of second control group data and the statistical model of double overall T test to obtain an evaluation result;
pushing the evaluation result to a user.
2. The method of claim 1, further comprising, prior to the acquiring the user data of the N experimental groups and the user data of the N control groups, respectively:
Acquiring respective identifications of a plurality of first users and respective identifications of a plurality of second users;
dividing the plurality of first users into the N experiment groups according to the respective identifications of the plurality of first users, and dividing the plurality of second users into the N experiment groups according to the respective identifications of the plurality of second users.
3. The method of claim 2, wherein the dividing the plurality of first users into the N experiment groups according to the respective identifications of the plurality of first users comprises:
determining N first hash values corresponding to the identifiers of the first users respectively;
dividing the plurality of first users into the N experiment groups according to the N first hash values.
4. The method of claim 2, wherein the dividing the plurality of second users into the N experiment groups according to the respective identities of the plurality of second users comprises:
determining N second hash values corresponding to the identifiers of the second users;
dividing the plurality of second users into the N control groups according to the N second hash values.
5. The method of claim 1, wherein evaluating the test product and the control product based on the N second test group data and the N second control group data to obtain an evaluation result comprises:
Selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer which is more than or equal to 1 and less than or equal to N;
and inputting the P second experimental group data and the P second control group data into a statistical model of double-population T test to obtain the evaluation result.
6. The method of claim 1, wherein evaluating the test product and the control product based on the N second test group data and the N second control group data to obtain an evaluation result comprises:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data;
randomly selecting N-1 different second control group data from the N second control group data to summarize until each second control group data is selected for N-1 times to obtain N third control group data;
and inputting the N third experimental group data and the N third control group data into a statistical model of double-overall T test to obtain the evaluation result.
7. The method of any one of claims 1-6, further comprising:
the N first experimental group data and the N first control group data are stored.
8. A product evaluation device, comprising:
the first acquisition module is used for respectively acquiring user data of N experimental groups and user data of N comparison groups according to each time period of M time periods, wherein the user data of each experimental group is user data obtained by at least one user using an experimental product, the user data of each comparison group is user data obtained by at least one user using a comparison product, M is an integer larger than 1, and N is an integer larger than 1;
the processing module is used for summarizing the user data of each experimental group for each time period to obtain N pieces of first experimental group data; summarizing the user data of each control group respectively to obtain N pieces of first control group data;
the evaluation module is used for respectively determining N groups of first experimental group data to be summarized and N groups of first control group data to be summarized, wherein each group of first experimental group data to be summarized comprises one first experimental group data on each of the M time periods, and each group of first control group data to be summarized comprises one first control group data on each of the M time periods;
Summarizing N groups of data of the first experimental group to be summarized respectively to obtain N second experimental group data, and summarizing N groups of data of the first comparison group to be summarized respectively to obtain N second comparison group data;
evaluating the experimental products and the control products according to the N pieces of second experimental group data, the N pieces of second control group data and the statistical model of double overall T test to obtain an evaluation result;
and the pushing module is used for pushing the evaluation result to the user.
9. The apparatus as recited in claim 8, further comprising:
the second acquisition module is used for acquiring the respective identifications of the plurality of first users and the respective identifications of the plurality of second users;
the dividing module is used for dividing the plurality of first users into the N experiment groups according to the respective identifications of the plurality of first users, and dividing the plurality of second users into the N experiment groups according to the respective identifications of the plurality of second users.
10. The apparatus of claim 9, wherein the partitioning module is specifically configured to:
determining N first hash values corresponding to the identifiers of the first users respectively;
Dividing the plurality of first users into the N experiment groups according to the N first hash values.
11. The apparatus of claim 9, wherein the partitioning module is further to:
determining N second hash values corresponding to the identifiers of the second users;
dividing the plurality of second users into the N control groups according to the N second hash values.
12. The apparatus of claim 8, wherein the evaluation module is further to:
selecting P pieces of second experiment group data from the N pieces of second experiment group data, and selecting P pieces of second comparison group data from the N pieces of second comparison group data, wherein P is an integer which is more than or equal to 1 and less than or equal to N;
and inputting the P second experimental group data and the P second control group data into a statistical model of double-population T test to obtain the evaluation result.
13. The apparatus of claim 8, wherein the evaluation module is further to:
randomly selecting N-1 different second experimental group data from the N second experimental group data to summarize until each second experimental group data is selected for N-1 times to obtain N third experimental group data;
Randomly selecting N-1 different second control group data from the N second control group data to summarize until each second control group data is selected for N-1 times to obtain N third control group data;
and inputting the N third experimental group data and the N third control group data into a statistical model of double-overall T test to obtain the evaluation result.
14. The apparatus according to any one of claims 8-13, further comprising:
and the storage module is used for storing the N first experimental group data and the N first control group data.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910819016.8A CN112446574B (en) | 2019-08-30 | 2019-08-30 | Product evaluation method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910819016.8A CN112446574B (en) | 2019-08-30 | 2019-08-30 | Product evaluation method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112446574A CN112446574A (en) | 2021-03-05 |
| CN112446574B true CN112446574B (en) | 2024-04-12 |
Family
ID=74735355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910819016.8A Active CN112446574B (en) | 2019-08-30 | 2019-08-30 | Product evaluation method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112446574B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112926690B (en) * | 2021-03-31 | 2023-09-01 | 北京奇艺世纪科技有限公司 | Data processing method, device, equipment and storage medium |
| CN113158497A (en) * | 2021-05-26 | 2021-07-23 | 中国平安人寿保险股份有限公司 | Online service experiment method and device, computer equipment and storage medium |
| CN113609013A (en) * | 2021-08-03 | 2021-11-05 | 挂号网（杭州）科技有限公司 | Evaluation method and device for AB test, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106447512A (en) * | 2016-09-13 | 2017-02-22 | 江西省农业科学院植物保护研究所 | Method of quantitative evaluation on control of natural growth of rice hopper population through field natural enemy |
| CN110033156A (en) * | 2018-12-14 | 2019-07-19 | 阿里巴巴集团控股有限公司 | A kind of determination method and device of business activity effect |
| US10387787B1 (en) * | 2015-10-28 | 2019-08-20 | Intuit Inc. | Method and system for providing personalized user experiences to software system users |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030049207A1 (en) * | 2001-09-10 | 2003-03-13 | The Procter & Gamble Company | Clinical method for evaluating the efficacy of caries prevention or treatment agents |
| EP2482946A1 (en) * | 2009-09-30 | 2012-08-08 | Zynga Inc. | Apparatuses, methods and systems for an engagement-tracking game modifier |
| US20160086121A1 (en) * | 2014-09-19 | 2016-03-24 | Benjamin Heilbrunn | Providing Gamification Analytics in an Enterprise Environment |
-
2019
- 2019-08-30 CN CN201910819016.8A patent/CN112446574B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10387787B1 (en) * | 2015-10-28 | 2019-08-20 | Intuit Inc. | Method and system for providing personalized user experiences to software system users |
| CN106447512A (en) * | 2016-09-13 | 2017-02-22 | 江西省农业科学院植物保护研究所 | Method of quantitative evaluation on control of natural growth of rice hopper population through field natural enemy |
| CN110033156A (en) * | 2018-12-14 | 2019-07-19 | 阿里巴巴集团控股有限公司 | A kind of determination method and device of business activity effect |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112446574A (en) | 2021-03-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11928432B2 (en) | Multi-modal pre-training model acquisition method, electronic device and storage medium | |
| CN112446574B (en) | Product evaluation method, device, electronic equipment and storage medium | |
| CN112306836B (en) | Sampling object determining method, apparatus, electronic device storage medium and program product | |
| CN110569969A (en) | Network model structure sampling method and device and electronic equipment | |
| CN112561332B (en) | Model management method, device, electronic equipment, storage medium and program product | |
| CN111461343A (en) | Model parameter updating method and related equipment thereof | |
| CN112015468B (en) | Interface document processing method and device, electronic equipment and storage medium | |
| CN111178937B (en) | User rewarding method and device for application, electronic equipment and readable storage medium | |
| CN111881339A (en) | Resource information pushing and informing method and device, electronic equipment and storage medium | |
| CN112015439B (en) | Embedding method, device, equipment and storage medium of user APP interest | |
| CN111026916B (en) | Text description conversion method and device, electronic equipment and storage medium | |
| CN111177479B (en) | Method and device for acquiring feature vector of node in relational network graph | |
| CN112580723A (en) | Multi-model fusion method and device, electronic equipment and storage medium | |
| CN112507249A (en) | Method and device for generating interest points, electronic equipment and storage medium | |
| CN111783872B (en) | Method, device, electronic equipment and computer readable storage medium for training model | |
| CN111638982B (en) | Flow guiding method, flow guiding device and electronic equipment | |
| CN111177558B (en) | Channel service construction method and device | |
| CN111523036B (en) | Search behavior mining method and device and electronic equipment | |
| CN112734454B (en) | User information determining method and device, electronic equipment and storage medium | |
| CN112446728B (en) | Advertisement recall method, device, equipment and storage medium | |
| CN111292223A (en) | Graph calculation processing method and device, electronic equipment and storage medium | |
| CN111651229A (en) | Font changing method, device and equipment | |
| CN111523000A (en) | Method, device, equipment and storage medium for importing data | |
| CN111782794A (en) | Question-answer response method and device | |
| CN111523664A (en) | Super-network parameter updating method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |