CN113689232B

CN113689232B - Method and device for detecting crowd recall service and electronic equipment

Info

Publication number: CN113689232B
Application number: CN202110851532.6A
Authority: CN
Inventors: 郝佳; 侯广宇; 王正
Original assignee: Alibaba Huabei Technology Co ltd
Current assignee: Alibaba Huabei Technology Co ltd
Priority date: 2021-07-27
Filing date: 2021-07-27
Publication date: 2023-07-28
Anticipated expiration: 2041-07-27
Also published as: CN113689232A

Abstract

The embodiment of the application discloses a method, a device and electronic equipment for detecting crowd recall services, wherein the method comprises the following steps: acquiring a first mapping relation, and determining at least one recallable crowd type according to the first mapping relation; filtering the at least one recallable crowd type from the full crowd type set, and determining at least one target crowd type to be detected; acquiring a second mapping relation, and determining a plurality of target users for the target crowd type according to the second mapping relation; and simulating and constructing a service call request by using the identifiers of the target users, and detecting whether the crowd recall service system can recall the target crowd type by calling the service provided by the crowd recall service system. Through the embodiment of the application, the real-time verification of the crowd recall service system can be effectively performed, and meanwhile, the influence of the real-time verification process on the actual online flow is reduced.

Description

Method and device for detecting crowd recall service and electronic equipment

Technical Field

The application relates to the technical field of crowd recall services, in particular to a method, a device and electronic equipment for detecting crowd recall services.

Background

The crowd recall service system is a system for providing crowd recall service, a requester can initiate crowd recall request to the crowd recall service system by taking user ID in a certain application system and the like as parameters, and correspondingly, the crowd recall service system can return crowd information to which the user belongs, for example, whether a certain user belongs to a crowd with high purchasing power, whether the user belongs to a crowd who has visited a certain shop in the last three days, and the like. The requesting party can use crowd information corresponding to specific users to conduct advertisement delivery and the like.

In the process of providing crowd recall service, quality assurance is also important, and accuracy of the crowd recall service system for providing service is required to be perceived in real time. Here, the correctness is to ensure that all people can be recalled correctly, that is, that any user ID can return the crowd information to which the user belongs. Thus, the trust degree of the advertisement owners and the like to the crowd recall service system is improved.

In order to achieve the purpose of the accuracy of the real-time perception of the accuracy of the external service provided by the crowd recall service system, a detection system for providing real-time detection service for the crowd recall service system can be provided. Because it needs to verify whether all people can be recalled correctly in the process of providing service to the outside in real time, one way can be: obtaining IDs of the whole users, constructing a request by utilizing the IDs of the users, simulating a delivery engine to send the request to a crowd recall service system, and if the crowd recall service system can return information of the crowd to which each user ID belongs, proving that the correctness of the crowd recall service system passes.

The foregoing manner can theoretically verify the correctness of the crowd recall service system, however, in practical application, since the magnitude of the total number of users is usually very large, for example, in a certain commodity object information system, the total number of users may be in the magnitude of billions, if request simulation is respectively performed on the total number of user IDs, and the request simulation is respectively sent to the crowd recall service system for processing, the processing task of the crowd recall service system will be very heavy; moreover, the detection process is performed simultaneously in the normal online service providing process of the crowd recall service system, and billions of analog requests can cause fluctuation of the online flow, so that the correctness of the crowd recall service system is difficult to verify in real time by using the scheme in practical application.

Therefore, how to effectively perform real-time verification on the crowd recall service system and reduce the influence of the real-time verification process on the flow on the actual line becomes a technical problem which needs to be solved by the technicians in the field.

Disclosure of Invention

The application provides a method, a device and electronic equipment for detecting crowd recall service, which can effectively check a crowd recall service system in real time and reduce the influence of a real-time check process on the flow on an actual line.

The application provides the following scheme:

a method of detecting crowd recall services, comprising:

acquiring a first mapping relation between a plurality of first user identifications and a first group type according to real-time log record information generated by a group recall service system in the service providing process, and determining at least one recall group type according to the first mapping relation;

filtering the at least one recallable crowd type from the total crowd type set, and determining at least one target crowd type to be detected according to a filtered result;

acquiring a second mapping relation between a plurality of second user identifiers and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;

And simulating and constructing a service call request by using the identifiers of the target users, and detecting whether the crowd recall service system can recall the target crowd type by calling the service provided by the crowd recall service system.

An apparatus for detecting crowd recall services, comprising:

the recall crowd type determining unit is used for acquiring a plurality of first mapping relations between the first user identifications and the first crowd types according to real-time log record information generated by the crowd recall service system in the service providing process, and determining at least one recall crowd type according to the first mapping relations;

the filtering unit is used for filtering the at least one recallable crowd type from the total crowd type set and determining at least one target crowd type to be detected according to a filtered result;

the target user determining unit is used for acquiring a second mapping relation between a plurality of second user identifications and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation;

And the service calling unit is used for simulating and constructing a service calling request by utilizing the identifiers of the plurality of target users, and detecting whether the crowd recall service system can recall the target crowd type by calling the service provided by the crowd recall service system.

A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the preceding claims.

An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of the preceding claims.

According to a specific embodiment provided by the application, the application discloses the following technical effects:

according to the embodiment of the application, as at least one recallable crowd type can be determined according to the real-time log record information generated by the crowd recall service system in the service providing process, namely, the part of crowd types can be checked through the real-time log record information, the part of crowd types can be filtered out from the whole crowd type set, and only the crowd types which can not be recalled can be further detected if the current real-time log record information is not determined. In this way, the number of types of target people that need to be detected is reduced. In addition, since it can obtain the specific target users corresponding to the target crowd types from the history log record information, it only needs to use the IDs of the target users to simulate and construct the service request and initiate the request to the crowd recall service, and then it can determine whether the target crowd types can be recalled according to the response information returned by the crowd recall service. In this way, only part of the target crowd types need to be detected by simulating the mode of constructing the service call request, and the target users with the historical mapping relation with the target crowd types can be directionally selected to construct the service call request, so that the number of the required construction and the sent service call requests can be greatly reduced, and the influence of a specific real-time detection process on the flow on the actual line of the crowd recall service is reduced.

Of course, not all of the above-described advantages need be achieved at the same time in practicing any one of the products of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 2 is a flow chart of a method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a process flow provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of an apparatus provided by an embodiment of the present application;

fig. 5 is a schematic diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

Firstly, it should be noted that, in order to facilitate understanding the technical solution provided in the embodiments of the present application, an application scenario of information delivery such as "advertisement" in merchandise object information is taken as an example, and a crowd recall service system and related crowd recall flow are simply introduced.

Referring to fig. 1, the commodity object information system 11 may mainly include a seller user (or a merchant user, etc.) and a buyer user, where the seller user may issue specific commodity object information through a virtual store on line opened by the commodity object information system, and the buyer user may browse the specific commodity object information and perform various operations such as collection, joining "shopping cart", and making a purchase.

In practical applications, the merchandise object information system may also generally have an associated information delivery system 12 for delivering information such as "advertisements" to users in the merchandise object information system 11. That is, the information specifically presented to the buyer user may include some "advertising" content in addition to the merchandise object information published by the specific seller user. In order to achieve the expected throwing effect, the directional throwing of people can be generally carried out. For example, an "advertisement" may need to be placed on a population with a relatively high purchasing power, such that a relatively high browse-purchase conversion is achieved, and so on.

To achieve the goal of group-oriented information delivery, the services provided by the specific group recall service system 13 may be used. Specifically, the crowd recall service system 13 may include a crowd construction system 131, and the crowd construction system 131 may add crowd labels to specific users by crowd construction, policy analysis, and the like. For example, specific crowd labels, such as high purchasing power, baby's, etc., are added to the user according to the data of the user's information, browsing, collecting, purchasing, ordering, etc., in the commodity object information system.

The information delivery system 12 may provide a delivery job configuration system 121 for a delivery job publisher such as an "advertiser" who may create a particular delivery job through the delivery job configuration system 121. In creating the delivery task, a variety of optional crowd labels may be provided according to crowd construction conditions of the particular crowd construction system 131. The advertiser can select the required crowd label according to the actual requirement. In this way, a specific task may include specific content to be delivered and crowd label information, that is, the corresponding content to be delivered needs to be delivered to the user with the corresponding crowd label. If there are a plurality of crowd labels selected by the advertiser, a logic relationship between the crowd labels may be set, for example, the user who is simultaneously associated with the crowd labels may be put in, or the user who has any crowd label may be put in. In addition, the "advertiser" may also select a particular delivery channel (also referred to as "ad spot") when submitting a delivery job. For example, a particular "ad spot" may include a live channel in the merchandise object information system 11, a particular search keyword, and so on. Thus, for the former, when the user accesses the live channel, whether to put a certain advertisement on the live channel is judged according to the crowd to which the specific user belongs; in the latter case, when a user initiates a search with a certain search keyword, whether to put a certain advertisement on the specific user is determined according to the crowd to which the user belongs, and so on.

That is, in particular, when the "advertisement" is put, it is determined whether or not to put the "advertisement" and what kind of advertisement is put on the user according to the situation in the process that the user actually accesses the commodity object information system 11. To this end, information delivery system 12 may also provide delivery engine 122, wherein delivery engine 122 may be multiple, e.g., different "ad spots" in the merchandise object information system may correspond to different product lines (e.g., including live broadcasts, searches, etc.), where different product lines may be provided with corresponding delivery engines 122, respectively. Such a delivery engine 122 may interface with a particular merchandise object information system 11. When a user accesses a certain product line in the commodity object information system, the commodity object information system may provide information such as the ID of the current visitor user to the corresponding delivery engine 122. Crowd recall service system 13 may also provide crowd recall service engine 132. After receiving the user ID provided by the merchandise object information system, the delivery engine 122 may initiate a request to the crowd recall service engine 132 with information such as the user ID and the corresponding channel identifier as parameters. Crowd recall service engine 132 may then determine and return crowd type information to which the user belongs. Thereafter, the placement engine 122 may determine whether or not to place an "advertisement" on a particular user based on the crowd type information to which the user belongs, and, if so, what kind of "advertisement" to place specifically.

For example, when a user searches for a product object information system using a keyword, the product object information system may provide information such as the user's ID to the delivery engine 122; the delivery engine 122 initiates a call request to the crowd recall service engine 132 by using the user's ID, channel, etc. information as parameters; after the crowd recall service engine 132 returns the crowd type information corresponding to the user, the delivery engine 122 delivers "advertisements" to the user according to the crowd type information to which the user specifically belongs. For example, in presenting search results through multiple resource bits in a returned search results page, the "advertisement" content may be presented at some of the resource bits, and so on.

It can be seen that, in a specific crowd recall flow, firstly, a user accesses the merchandise object information system 11, in this process, the merchandise object information system 11 provides user ID information to the delivery engine 122, the delivery engine 122 initiates a request to the crowd recall service engine 132 according to the user ID, the crowd recall service engine 132 can determine crowd type information to which the user belongs, and returns the crowd type information to the delivery engine 122, and the delivery engine 122 determines an "advertisement" delivery policy for the user according to the crowd type information to which the user belongs.

The crowd recall service system 13 may include information obtained in a history crowd construction process and information constructed in real time when providing crowd recall service, and may provide crowd information to which a specific user belongs after combining two pieces of information when in specific implementation. For example, a user belongs to a high-purchasing-power crowd, and the crowd label can be determined according to information such as historical purchasing records of the user; at the same time, the user has accessed a store three days recently, so the tag may be obtained by real-time analysis after receiving a specific crowd recall service request, and so on.

In summary, in the crowd recall service system 13, the constructed crowd types may be very large, and whether all the crowd can be recalled correctly in the process of providing service to the outside is an important standard for checking the service quality of the crowd recall service system 13. That is, it is not known in advance when a specific user accesses the merchandise object information system, so it is necessary to ensure that when a crowd recall request for any user is received at any time, the crowd to which the user belongs can be accurately returned, so that a reliable service can be truly provided for an "advertiser", and trust of the "advertiser" and the like to the crowd recall service system 13 is obtained. Because the visitor user in the specific merchandise object information system has randomness, it is necessary to provide a detection system 14, and the detection system 14 can provide real-time online detection service for the crowd recall service system 13, so as to determine the accuracy and reliability of the crowd recall service system 13 for providing service.

However, in the prior art, since the mapping relationship between the specific user IDs and the belonging people is not known in the specific detection system 14, in order to check whether the crowd recall service system 13 can perform correct recall on all people, as described in the background section, only the IDs of the total number of users can be obtained, and then the user IDs are used to construct a request, and the simulated delivery engine 122 sends the request to the crowd recall service engine 132. However, since the above-described detection process is required during normal online out-of-line service of the crowd recall service engine 132, if a substantial amount of simulation requests are made, the normal flow of the crowd recall service system 13 is severely affected.

In view of the above, in the embodiment of the present application, the real-time log record information generated by the crowd recall service system 13 in the process of providing the service to the outside may be acquired, and according to this real-time log record information, the request parameters associated with the plurality of service call requests received by the crowd recall service system 13 and the response parameters associated with the corresponding response messages may be determined within the target detection period (for example, within the last N minutes, etc.). The request parameters include user identifiers such as user IDs, and the corresponding response parameters include crowd type information matched by the crowd recall service system 13 for the user IDs. That is, from such real-time log record information, the mapping relationship between the user ID and the crowd type can be obtained. Of course, since not all users will access the application systems such as the merchandise object information system 11 in the last N minutes, only the user IDs of part of the users and the crowd types are included in the mapping relationship determined in the real-time log record information. On the other hand, since the real-time logging information within the last N minutes can basically reflect the real-time service situation of the crowd recall service system 13, the crowd type in the above mapping relationship can be determined as the first crowd type (may be one or more, and typically a plurality) that can be recalled. That is, since the mapping relationship between the part of user IDs and the crowd type can be obtained from the real-time log of the last N minutes, it can be proved that at least in the last N minutes, the part of crowd type is normally recallable, and therefore, in the embodiment of the present application, the part of crowd type can be determined as a recallable crowd type. In addition, since the mapping relationship is determined from the recently acquired real-time log record information, the recalled crowd type can be considered correct as long as the format and the like are in line with expectations.

Since the at least one recallable crowd type has been verified as being recallable and correct, the recallable crowd types may be filtered out of a full set of crowd types (the information may be pre-acquired) and at least one target crowd type to be detected may be determined based on the filtered results. That is, only the crowd type that is not recalled in the real-time log record information may need to be further checked by constructing a service call request, whether the crowd recall service system can recall the part of the target crowd type. In particular, some invalid, expired and target list-hit crowd types and the like can be filtered from the total crowd type set, so that the number of crowd types needing to be detected is further reduced.

For the target crowd type to be detected, the identification of a plurality of target users having a mapping relationship with the target crowd type may be acquired according to history log information (for example, the past 48 hours, etc.) generated by the crowd recall service system 13 in the process of providing services to the outside. That is, a user may not access a merchandise object information system 11 within the past N minutes, but may have accessed the merchandise object information system 11 at a time within the past 48 hours, and the associated delivery engine 122 once called the crowd recall service with the user's ID as a parameter, at this time, there will be a corresponding request and response record in the history log record information, and the mapping relationship between the user ID and the crowd label may be determined therefrom.

That is, for some target crowd types to be detected, it may be determined from the history log record, which target users have a mapping relationship with the target crowd types, so that service call requests may be constructed by using the identifiers of the target users and sent to the crowd recall service system 13, and according to the response result of the crowd recall service system 13, it may be detected whether the crowd recall service system 13 can recall the target crowd types.

In this way, since at least one recallable crowd type can be determined from the real-time logging information and filtered out of the full population of crowd types sets, only at least one target crowd type that fails to be determined from the current real-time logging information needs to be further detected, and thus the detected crowd type is reduced, thereby reducing the magnitude of the service request that needs to be constructed. In addition, since it can obtain the specific target users corresponding to the target crowd types from the history log record information, it only needs to use the IDs of the target users to simulate and construct the service request and initiate the request to the crowd recall service, and then it can determine whether the target crowd types can be recalled according to the response information returned by the crowd recall service.

The implementation scheme provided by the embodiment of the application is described in detail below.

Examples

First, this embodiment, from the perspective of the detection system 14 depicted in FIG. 1, provides a method of verifying crowd recall services, see FIG. 2, which may include:

s201: according to real-time log record information generated by the crowd recall service system in the service providing process, a first mapping relation between a plurality of first user identifications and first crowd types is obtained, and at least one recall crowd type is determined according to the first mapping relation.

In the crowd recall service system, a log center system is usually associated, the log center can be a log management system integrating log acquisition, transmission and storage, and log data can be conveniently stored in real time through the log center. In the embodiment of the application, the type of a part of the recallable crowd can be determined by recording information in real time. Specifically, a relatively short detection period may be set, for example, assuming that the detection period may be one minute, the specific real-time log may specifically refer to log record information collected in the last one minute. And then, determining a part of the group types which can be recalled by using the log record information acquired in the last minute. When the next detection period comes, the log record information collected in the last minute is continuously taken as real-time log record information, and the like.

The specific real-time log record information may include the following contents: the received request parameters associated with the plurality of service call requests and the response parameters associated with the corresponding response messages in the current detection period. The specific request parameter may include an identifier of the first user, and the response parameter includes information of a first group type, so that a first mapping relationship between a plurality of first user identifiers and the first group type may be obtained according to the identifier of the first user included in the specific request parameter and the first group type information included in the corresponding response parameter. The first mapping relationship belongs to a real-time mapping relationship. For this portion of the first crowd-type, it may prove to be recallable because of the real-time recall. In addition, as long as the format and the like meet the expectations, the correctness of the recall result can be proved, and no additional detection is needed. Therefore, it is no longer necessary to detect whether the part of crowd type can be recalled by means of simulating the construction of a recall request.

S202: filtering the at least one recallable crowd type from the total crowd type set, and determining at least one target crowd type to be detected according to the filtered result.

After determining the type of the group which can be recalled according to the real-time log record information, the group is not required to be recalled when the type of the group is verified, and whether the group can be recalled is checked by sending a calling request to the group recall service. Thus, the portion of the population types may be filtered out from the full population of population type sets. The remaining crowd types are not verified in the current real-time log record information, so that verification can be performed by sending a call request to the crowd recall service.

The crowd type of the whole crowd can be obtained by querying a crowd construction system and the like, and in specific implementation, the crowd construction system can push real-time information of the whole crowd type to a detection system, or the detection system can pull the information from the crowd construction system and the like.

In practical applications, the total population type may be generally large, for example, in a commodity object information system, the total population type may include millions of population types. In the embodiment of the application, by filtering the crowd types already checked in the real-time log record information, the number of the remaining target crowd types to be detected is greatly reduced.

In addition, in practical applications, the total population type set may further include some population types that expire, fail, and/or hit some target lists (e.g., blacklist, switch traffic gray list, on-the-spot list, etc.), and the population types may also be filtered out, so as to further reduce the number of target population types that need to be detected.

That is, as shown in fig. 3, a funnel-type crowd type filtering scheme may be provided in the specific implementation, and by performing layer-by-layer filtering on the total crowd types, the number of target crowd types that need to be detected is greatly reduced.

S203: and acquiring a second mapping relation between a plurality of second user identifiers and a second crowd type according to historical log record information generated by the crowd recall service system in the service providing process, and determining a plurality of target users for the target crowd type according to the second mapping relation.

After determining the type of the target crowd to be detected, a second mapping relationship between a plurality of second user identifiers and the second crowd type can be obtained from the history log record information. Such a second mapping relationship belongs to the history mapping relationship. The history log information may be log information of a longer period of time, for example, log information of the last 48 hours, or the like. The method comprises the steps of including request parameters associated with a plurality of historical service call requests and response parameters associated with corresponding historical response messages, and further determining a mapping relation between the identification of the second user and the second crowd type according to the identification of the second user included in the request parameters and the information of the second crowd type included in the corresponding response parameters. Because such history log information may involve a longer time span, which may include more user identifications, and more crowd types, which may include the target crowd types to be detected, it is possible to learn which users have a mapping relationship with those target crowd types. These users may in turn be determined as target users for the mock construct call service request.

That is, since the mapping relationship between the user and the crowd type is known, when the detection of a part of the crowd type is required, the user identification having the mapping relationship with the crowd type can be directly utilized to perform the simulation of the call service request. In particular, all users having a mapping relation with the target crowd type in the history log record information can be determined as target users, or part of users can be selected from a plurality of users corresponding to the same target crowd type as target users, so that the number of call requests to be sent is further reduced. For example, after practical testing, the number of the target crowd types to be detected can be reduced to one thousandth of the number of the total crowd types or even lower after the filtering is performed in the various modes. That is, the number of millions of types of people to be detected can be reduced by thousands.

S204: and simulating and constructing a service call request by using the identifiers of the target users, and detecting whether the crowd recall service system can recall the target crowd type by calling the service provided by the crowd recall service system.

After a specific target user is determined, a service call request can be simulated and constructed by using user identifications such as ID of the target user, and then the simulated service call request can be sent to a crowd recall service system for calling crowd recall services. And detecting whether the crowd recall service system can recall the target crowd type or not through response information returned by the crowd recall service. For example, if a certain target crowd type is included in the response information returned by the crowd recall service, the crowd recall service may prove that the target crowd type can be recalled.

In a specific implementation, a request related to a user identifier may occur, where a crowd recall service fails to recall a crowd type corresponding to the crowd type, for example, a certain target crowd type is "a store has been accessed in three days recently", and it is determined that a target user corresponding to the target crowd type has a user A, B, C through historical log record information; after the crowd recall service call requests are respectively constructed by using the information such as the IDs of the users and the like and sent to the crowd recall service system, the crowd recall service system may not comprise the target crowd type aiming at the crowd type possibly returned by the user A. However, since there is also a user B, C for checking the target crowd type, the returned crowd type including the target crowd type can be determined to be recalled normally as long as the request is constructed for other users such as the user B, C.

In addition, in practical applications, because the total number of crowd types may be relatively large, for example, in the millions as described in the foregoing examples, a particular crowd recall service system may divide a particular crowd recall service into multiple sub-services, and deploy the sub-services in multiple different rooms, each of which may provide recall services for some of the crowd types therein. For example, assuming a total of 100 thousands of people types, 10 sub-services may be partitioned, deployed in 10 different rooms, each providing recall services for 10 of the thousands of people types, and so on. Wherein, each sub-service is corresponding to the crowd types respectively, and can be configured by the crowd recall service system.

In this case, particularly when a service provided by the crowd recall service system is invoked, since a target crowd type corresponding to a specific target user is known, service configuration information of the crowd recall service system can also be obtained, and a corresponding relationship between the sub-service and the crowd type can also be obtained according to the service configuration information. In this way, the sub-service corresponding to the target crowd type can be invoked according to the corresponding relation. For example, for a service call request constructed according to an ID of a certain target user, when the request is sent, a sub-service corresponding to a target crowd type corresponding to the target user may be determined first according to the target crowd type corresponding to the target user, and then the sub-service may be called by using the request. By the method, the group recall service can be actively detected in the machine room.

In particular implementations, the service configuration information of the specific crowd recall service system may be dynamically changed, that is, the correspondence between each sub-service and the specific crowd type may be changed in real time. Therefore, the detection system can also acquire the service configuration information of the crowd recall service system in real time, so as to acquire the latest corresponding relation between the sub-service and the crowd type.

By the method, as long as the specific crowd type is included in the response returned by the specific crowd recall service, the target crowd type can be determined to be recalled normally. In addition, in the specific implementation, the correctness of the specific recalled crowd type can be checked. Specifically, in the embodiment of the present application, since the mapping relationship between the plurality of second user identifiers and the second crowd type may be determined through the history log record information in advance, the mapping relationship may be used as the expected result. And after the constructed service calling request is used for calling the service provided by the crowd recall service system, determining the mapping relation between the target user and the third crowd type as an actual result according to the response result. In this way, the correctness of the recall result can be checked by judging whether the actual result is consistent with the expected result, that is, whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.

For example, the target user determined for a certain target crowd type 1 according to the history log record information includes a user a, that is, a mapping relationship exists between the user a and the target crowd type 1. After the user A identifier is used for constructing a service call request and sending the service call request to the crowd recall service system, if the corresponding crowd type is crowd type 1, the crowd type recalled by the specific crowd recall service system can be proved to be correct.

Of course, in practical applications, there may be a case where the third crowd type corresponding to a certain user identifier is inconsistent with the second crowd type. This may be due to a recall result error from the crowd recall service system, a change in crowd construction results from the crowd construction system, and so on. Therefore, when the inconsistency occurs, the crowd construction system can be queried to determine the fourth crowd type corresponding to the user identifier. If the third crowd type is consistent with the fourth crowd type, the correctness check of the recall result can be determined to pass.

After the detection of the multiple target crowd types is completed, the types of the people which cannot be recalled can be counted and summarized, and then, the attribution analysis can be carried out on the specific types of the people which cannot be recalled, so that the specific reasons of the non-recall can be analyzed. After the attribution analysis is completed, specific problems can be automatically repaired according to specific analysis results, so that self-healing is realized; for the problem that the automatic repair is not possible, early warning information can be provided, and the like.

In summary, since at least one recallable crowd type can be determined according to the real-time log record information generated by the crowd recall service system in the process of providing services, that is, the part of crowd types can be checked through the real-time log record information, the part of crowd types can be filtered out from the whole crowd type set, so that only the crowd types which can not be recalled are required to be further detected if the recallable crowd types cannot be determined from the current real-time log record information. In this way, the number of types of target people that need to be detected is reduced. In addition, since it can obtain the specific target users corresponding to the target crowd types from the history log record information, it only needs to use the IDs of the target users to simulate and construct the service request and initiate the request to the crowd recall service, and then it can determine whether the target crowd types can be recalled according to the response information returned by the crowd recall service. In this way, only part of the target crowd types need to be detected by simulating the mode of constructing the service call request, and the target users with the historical mapping relation with the target crowd types can be directionally selected to construct the service call request, so that the number of the required construction and the sent service call requests can be greatly reduced, and the influence of a specific real-time detection process on the flow on the actual line of the crowd recall service is reduced.

It should be noted that, in the embodiments of the present application, the use of user data may be involved, and in practical applications, user specific personal data may be used in the schemes described herein within the scope allowed by applicable legal regulations in the country where the applicable legal regulations are met (for example, the user explicitly agrees to the user to actually notify the user, etc.).

Corresponding to the foregoing method embodiment, the embodiment of the present application further provides an apparatus for detecting a crowd recall service, referring to fig. 4, where the apparatus may include:

a recall population type determining unit 401, configured to obtain a first mapping relationship between a plurality of first user identifiers and a first population type according to real-time log record information generated by a population recall service system in a service providing process, and determine at least one recall population type according to the first mapping relationship;

a filtering unit 402, configured to filter the at least one recallable crowd type from the total crowd type set, and determine at least one target crowd type to be detected according to a filtered result;

a target user determining unit 403, configured to obtain second mapping relationships between a plurality of second user identifiers and a second crowd type according to history log record information generated by the crowd recall service system in a service providing process, and determine a plurality of target users for the target crowd type according to the second mapping relationships;

And a service calling unit 404, configured to simulate and construct a service calling request by using the identifiers of the multiple target users, and detect whether the crowd recall service system can recall the target crowd type by calling the service provided by the crowd recall service system.

Specifically, the recall population type determining unit may specifically be configured to:

according to the real-time log record information, determining request parameters associated with a plurality of service call requests received in the current detection period and response parameters associated with corresponding response messages;

and acquiring a first mapping relation between the plurality of first user identifications and the first person group type according to the first user identifications included in the request parameters and the information of the first person group type included in the corresponding response parameters.

In addition, the filter unit may also be used to:

and filtering out the crowd types which expire, lose efficacy and/or hit a target list from the total crowd type set, so as to determine the target crowd type to be detected.

The target user determination unit may specifically be configured to:

acquiring request parameters associated with a plurality of historical service call requests and response parameters associated with corresponding historical response messages according to the historical log record information;

And determining a mapping relation between the identification of the second user and the second crowd type according to the identification of the second user included in the request parameter and the information of the second crowd type included in the corresponding response parameter.

The crowd recall service system is used for providing a crowd type recall service for a user, wherein the crowd recall service system provides a service comprising a plurality of sub-services which are respectively deployed in a plurality of different servers, and each sub-service is used for providing a part of crowd type recall service;

at this time, the service calling unit may specifically be configured to:

acquiring service configuration information of the crowd recall service system, and acquiring a corresponding relation between the sub-service and crowd types according to the service configuration information;

and calling the sub-service corresponding to the target crowd type according to the corresponding relation.

Wherein the service configuration information may be dynamically changed;

at this time, the service calling unit may specifically be configured to:

and acquiring service configuration information of the crowd recall service system in real time.

In addition, the apparatus may further include:

the response result determining unit is used for determining a third crowd type which is recalled by the target user in the returned response result after the service provided by the crowd recall service system is called by the constructed service call request;

And the correctness checking unit is used for checking the correctness of the recall result by judging whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.

In addition, the apparatus may further include:

the query unit is used for determining a fourth crowd type corresponding to a user identifier by querying the crowd construction system if the third crowd type corresponding to the user identifier is inconsistent with the second crowd type; and if the third crowd type is consistent with the fourth crowd type, determining that the correctness check of the recall result is passed.

Furthermore, the apparatus may further include:

and the summarization result providing unit is used for determining the type of the person who cannot be recalled after the detection of whether the group recall service system can recall the target group type is finished, and providing a summarization result of the type of the person who cannot be recalled.

In addition, the apparatus may further include:

an attribution analysis unit for carrying out attribution analysis for the group types of the non-recall people;

and the repair early warning unit is used for repairing the problems in the crowd recall service system according to the attribution analysis result or providing early warning information.

In addition, the embodiment of the application further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the method of any one of the foregoing method embodiments.

And an electronic device comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of the preceding method embodiments.

Fig. 5 illustrates an architecture of an electronic device, which may include a processor 510, a video display adapter 511, a disk drive 512, an input/output interface 513, a network interface 514, and a memory 520, among others. The processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, and the memory 520 may be communicatively coupled via a communication bus 530.

The processor 510 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., for executing relevant programs to implement the technical solutions provided in the present application.

The Memory 520 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory ), static storage device, dynamic storage device, or the like. The memory 520 may store an operating system 521 for controlling the operation of the electronic device 500, and a Basic Input Output System (BIOS) for controlling the low-level operation of the electronic device 500. In addition, a web browser 523, a data storage management system 524, a service detection processing system 525, and the like may also be stored. The service detection processing system 525 may be an application program that specifically implements the operations of the foregoing steps in the embodiments of the present application. In general, when the technical solutions provided in the present application are implemented by software or firmware, relevant program codes are stored in the memory 520 and invoked by the processor 510 to be executed.

The input/output interface 513 is used for connecting with an input/output module to realize information input and output. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.

The network interface 514 is used to connect communication modules (not shown) to enable communication interactions of the device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).

Bus 530 includes a path to transfer information between components of the device (e.g., processor 510, video display adapter 511, disk drive 512, input/output interface 513, network interface 514, and memory 520).

It should be noted that although the above devices only show the processor 510, the video display adapter 511, the disk drive 512, the input/output interface 513, the network interface 514, the memory 520, the bus 530, etc., in the specific implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the present application, and not all the components shown in the drawings.

From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general purpose hardware platform. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in the embodiments or some parts of the embodiments of the present application.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a system or system embodiment, since it is substantially similar to a method embodiment, the description is relatively simple, with reference to the description of the method embodiment being made in part. The systems and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

The above method, device and electronic equipment for detecting crowd recall services provided by the present application are described in detail, and specific examples are applied to illustrate the principles and embodiments of the present application, and the above examples are only used to help understand the method and core ideas of the present application; also, as will occur to those of ordinary skill in the art, many modifications are possible in view of the teachings of the present application, both in the detailed description and the scope of its applications. In view of the foregoing, this description should not be construed as limiting the application.

Claims

1. A method of detecting crowd recall services, comprising:

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

The obtaining a first mapping relationship between the plurality of first user identifications and the first person group type includes:

3. The method as recited in claim 1, further comprising:

4. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the obtaining a second mapping relationship between the plurality of second user identifiers and the second crowd type includes:

5. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the crowd recall service system provides services comprising a plurality of sub-services which are respectively deployed in a plurality of different servers, wherein each sub-service is used for providing recall services of part of crowd types;

the calling of the service provided by the crowd recall service system comprises the following steps:

6. The method of claim 5, wherein the step of determining the position of the probe is performed,

the service configuration information is dynamically changed;

the obtaining the service configuration information of the crowd recall service system includes:

7. The method as recited in claim 1, further comprising:

After the constructed service calling request is used for calling the service provided by the crowd recall service system, determining a third crowd type which is recalled by the target user in a returned response result;

and checking the correctness of the recall result by judging whether the third crowd type corresponding to the same user identifier is consistent with the second crowd type.

8. The method as recited in claim 7, further comprising:

if the third crowd type corresponding to a certain user identifier is inconsistent with the second crowd type, determining a fourth crowd type corresponding to the user identifier by querying a crowd construction system;

and if the third crowd type is consistent with the fourth crowd type, determining that the correctness check of the recall result is passed.

9. The method as recited in claim 1, further comprising:

after the detection of whether the crowd recall service system can recall the target crowd type is completed, determining the crowd type which cannot be recalled, and providing a summarizing result of the crowd type which cannot be recalled.

10. The method as recited in claim 9, further comprising:

performing attribution analysis on the group types of the non-recall people;

And repairing the problems in the crowd recall service system according to the attribution analysis result, or providing early warning information.

11. An apparatus for detecting crowd recall services, comprising:

12. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method of any of claims 1 to 10.

13. An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of claims 1 to 10.