CN115907272B

CN115907272B - Method and device for evaluating brokers, electronic equipment and storage medium

Info

Publication number: CN115907272B
Application number: CN202211506444.3A
Authority: CN
Inventors: 武晓飞; 陈开江
Original assignee: Seashell Housing Beijing Technology Co Ltd
Current assignee: Seashell Housing Beijing Technology Co Ltd
Priority date: 2022-11-29
Filing date: 2022-11-29
Publication date: 2023-11-14
Anticipated expiration: 2042-11-29
Also published as: CN115907272A

Abstract

The embodiment of the disclosure discloses a method and a device for evaluating a broker, electronic equipment and a storage medium, wherein the method comprises the following steps: obtaining a plurality of session data of interaction of at least one broker and at least one user aiming at least one object in a set duration to obtain a session set; determining a first evaluation score of the preset broker based on a plurality of first session data corresponding to the preset broker in the session set; determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to the preset object in the session set; determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to the preset user in the session set; determining a capacity of the preset broker based on at least one of the first, second, and third rating scores; the embodiment realizes comprehensive evaluation of the brokers and solves the problem that the prior art is based on manual evaluation.

Description

Method and device for evaluating brokers, electronic equipment and storage medium

Technical Field

The present disclosure relates to computer vision technology, and more particularly, to a method and apparatus for evaluating a broker, an electronic device, and a storage medium.

Background

In the existing intelligent assistant scene, the diagnosis of the session is based on the diagnosis of a single session, the global diagnosis target is not available, and the diagnosis in the single session diagnosis is summarized through experience; on the premise that the evaluation of the session is based on a single session, the evaluation of multiple aspects of the broker cannot be realized, and the evaluation of the broker obtained based on the single session is also from human judgment.

Disclosure of Invention

The present disclosure has been made in order to solve the above technical problems. The embodiment of the disclosure provides a method and a device for evaluating a broker, electronic equipment and a storage medium.

According to an aspect of an embodiment of the present disclosure, there is provided a method of evaluating a broker, including:

obtaining a plurality of session data of interaction of at least one broker and at least one user aiming at least one object in a set duration to obtain a session set; wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects;

determining a first evaluation score of a preset broker based on a plurality of first session data corresponding to the preset broker in the session set;

Determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set; wherein the plurality of second session data corresponds to a plurality of the brokers;

determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to a preset user in the session set; wherein the plurality of third session data corresponds to a plurality of the brokers;

the capacity of the preset broker is determined based on at least one of the first, second, and third rating scores.

Optionally, the determining the first evaluation score of the preset broker based on the plurality of first session data of the corresponding preset broker in the session set includes:

determining at least one conversation sentence corresponding to the preset broker in the plurality of conversation data;

determining a first base score for the preset broker based on a number of first preset words and second preset words included in the at least one conversational sentence;

determining an expression score of the preset broker based on a number of third preset words included in the at least one conversational sentence;

The first evaluation score of the preset broker is determined based on the first base score and the expression score.

Optionally, the determining the first rating score of the preset broker based on the first base score and the expression score includes:

determining a performance change rate of the preset broker, and determining a corresponding performance change rate ranking of the preset broker in a broker set to which the preset broker belongs; the broker set includes a plurality of brokers;

determining a second base score based on the performance change rate, the ranking of performance change rates, and the first base score;

the first evaluation score of the preset broker is determined based on the second base score and the expression score.

Optionally, the determining the performance change rate of the preset broker and determining the ranking of the performance change rates of the preset broker in the set of brokers to which the preset broker belongs include:

determining a performance change rate of the preset broker based on an average performance of the preset broker within each sub-preset duration within the preset duration; wherein the preset duration comprises a plurality of sub-preset durations;

And determining a performance change rate ranking corresponding to the preset broker based on a plurality of performance change rates corresponding to a plurality of brokers included in the preset broker-to-broker collection.

Optionally, the determining, based on the plurality of second session data corresponding to the preset object in the session set, a second evaluation score of the preset broker includes:

determining an adoption score corresponding to the preset broker based on the unadopted times of the preset broker on the recommended sentences in the second session data;

determining a response rate score corresponding to the preset broker based on the explanation response rate of the preset broker to the preset object in the second session data;

determining a solution rate score corresponding to the preset broker based on the solution rates of the preset broker in the second session data;

determining the second valuation score for the preset broker based on the adoption score, the return score, and the answer score.

Optionally, the determining, based on the number of times the preset broker does not take the recommended sentence in the second session data, a taking score corresponding to the preset broker includes:

Obtaining n recommended sentences of the recommended sentences which are not adopted by the preset brokers in the second session data and a recommended sentence subset of the recommended sentences which are adopted by the brokers in the second session data; wherein n is equal to or greater than zero;

determining the unadopted times of the preset broker based on the intersection between the n recommended sentences and the recommended sentence subsets;

and determining the adoption score corresponding to the preset broker based on the unadopted times.

Optionally, the determining, based on the explanation response rate of the preset broker to the preset object in the second session data, a response rate score corresponding to the preset broker includes:

determining the explanation times and the replied times of the explanation of the preset object by the plurality of brokers in the plurality of second session data, and determining a plurality of explanation replying rates corresponding to the plurality of brokers; wherein each of the brokers corresponds to one of the explanation replies;

and determining the response rate score corresponding to the preset broker based on the ranking of the explanation response rate corresponding to the preset broker in the plurality of explanation response rates.

Optionally, the determining, based on the answer rates of the preset brokers in the second session data, the answer rate score corresponding to the preset broker includes:

determining the number of questions asked by the plurality of users and the number of questions answered by the plurality of brokers to the plurality of users in the plurality of second session data, and determining a plurality of answer rates corresponding to the plurality of brokers; wherein each of the brokers corresponds to one of the solution rates;

determining the answer rate score corresponding to the preset broker based on the ranking of the answer rates corresponding to the preset broker in the plurality of answer rates.

Optionally, the determining, based on the plurality of third session data of the corresponding preset user in the session set, a third evaluation score of the preset broker includes:

determining a recommendation rate score corresponding to the preset broker based on the recommendation rates corresponding to the preset broker in the plurality of third session data;

determining a label recommendation score corresponding to the preset broker based on the recommended label sequences of the plurality of brokers in the plurality of third session data;

and determining a third evaluation score of the preset broker based on the recommendation rate score and the tag recommendation score.

Optionally, the determining, based on the recommendation rates corresponding to the preset brokers in the third session data, a recommendation rate score corresponding to the preset broker includes:

based on the third session data, obtaining a first number of times each of a plurality of brokers recommends the object for the preset user, and a first number of replies to each of the brokers by the preset user;

determining a plurality of recommendation rates based on the plurality of first times and the plurality of first reply times;

and determining the recommendation rate score corresponding to the preset broker based on the ranking of the recommendation rates corresponding to the preset broker in the plurality of recommendation rates.

Optionally, the determining, based on the recommended tag orders of the plurality of third session data corresponding to the plurality of brokers, a tag recommendation score corresponding to the preset broker includes:

determining a plurality of ranking assessment indicator scores corresponding to the plurality of brokers based on a recommended tag order of the plurality of brokers in the plurality of third session data; wherein each of the brokers corresponds to one of the rank evaluation index scores;

And determining the label recommendation score corresponding to the preset broker based on the ranking assessment index score corresponding to the preset broker.

Optionally, the method further comprises:

at least one hint information to the preset broker in response to at least one of the first, second, and third rating scores meeting a preset condition.

According to another aspect of the embodiments of the present disclosure, there is provided an evaluation apparatus of a broker, including:

the data acquisition module is used for acquiring a plurality of session data of interaction between at least one broker and at least one user aiming at least one object in a set duration to acquire a session set; wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects;

a first evaluation module for determining a first evaluation score of a preset broker based on a plurality of first session data corresponding to the preset broker in the session set;

a second evaluation module for determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set; wherein the plurality of second session data corresponds to a plurality of the brokers;

A third evaluation module for determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to a preset user in the session set; wherein the plurality of third session data corresponds to a plurality of the brokers;

and a capacity determining module for determining a capacity of the preset broker based on at least one of the first, second, and third rating scores.

Optionally, the first evaluation module includes:

a sentence screening unit for determining at least one conversation sentence corresponding to the preset broker in the plurality of conversation data;

a first base score unit for determining a first base score of the preset broker based on the number of first preset words and second preset words included in the at least one conversational sentence;

an expression scoring unit for determining an expression score of the preset broker based on a number of third preset words included in the at least one conversational sentence;

and a first evaluation score unit configured to determine the first evaluation score of the preset broker based on the first base score and the expression score.

Optionally, the first evaluation score unit is specifically configured to determine a performance change rate of the preset broker, and determine a ranking of corresponding performance change rates of the preset broker in a broker set to which the preset broker belongs; the broker set includes a plurality of brokers; determining a second base score based on the performance change rate, the ranking of performance change rates, and the first base score; the first evaluation score of the preset broker is determined based on the second base score and the expression score.

Optionally, the first evaluation score unit is configured to determine, when determining a performance change rate of the preset broker and determining a ranking of the performance change rates of the preset broker in a corresponding broker set to which the preset broker belongs, a performance change rate of the preset broker based on an average performance of the preset broker in each sub-preset duration in the preset duration; wherein the preset duration comprises a plurality of sub-preset durations; and determining a performance change rate ranking corresponding to the preset broker based on a plurality of performance change rates corresponding to a plurality of brokers included in the preset broker-to-broker collection.

Optionally, the second evaluation module includes:

an adoption scoring unit, configured to determine an adoption score corresponding to the preset broker based on the number of times the preset broker does not adopt the recommended sentences in the plurality of second session data;

a reply rate scoring unit, configured to determine a reply rate score corresponding to the preset broker based on the explanation reply rate of the preset broker to the preset object in the plurality of second session data;

a solution rate scoring unit, configured to determine a solution rate score corresponding to the preset broker based on the solution rates of the preset broker in the plurality of second session data;

and a second evaluation score unit configured to determine the second evaluation score of the preset broker based on the adoption score, the return rate score, and the answer rate score.

Optionally, the adoption scoring unit is specifically configured to obtain n recommended sentences of the plurality of second session data, where the recommended sentences are not adopted by the preset broker, and a subset of recommended sentences of the plurality of second session data, where the subset of recommended sentences is adopted by the plurality of brokers; wherein n is equal to or greater than zero; determining the unadopted times of the preset broker based on the intersection between the n recommended sentences and the recommended sentence subsets; and determining the adoption score corresponding to the preset broker based on the unadopted times.

Optionally, the reply rate scoring unit is specifically configured to determine the number of times that the plurality of brokers in the plurality of second session data explain the preset object and the number of times that the explanation is replied to, and determine a plurality of explanation reply rates corresponding to the plurality of brokers; wherein each of the brokers corresponds to one of the explanation replies; and determining the response rate score corresponding to the preset broker based on the ranking of the explanation response rate corresponding to the preset broker in the plurality of explanation response rates.

Optionally, the answer rate scoring unit is specifically configured to determine a number of times of questions asked by the plurality of users and a number of times of questions answered by the plurality of brokers to the plurality of users in the plurality of second session data, and determine a plurality of answer rates corresponding to the plurality of brokers; wherein each of the brokers corresponds to one of the solution rates; determining the answer rate score corresponding to the preset broker based on the ranking of the answer rates corresponding to the preset broker in the plurality of answer rates.

Optionally, the third evaluation module includes:

a recommendation rate scoring unit, configured to determine a recommendation rate score corresponding to the preset broker based on recommendation rates corresponding to the preset broker in the plurality of third session data;

An index scoring unit, configured to determine a tag recommendation score corresponding to the preset broker based on a recommended tag order corresponding to the plurality of brokers in the plurality of third session data;

and a third evaluation score unit configured to determine a third evaluation score of the preset broker based on the recommendation rate score and the tag recommendation score.

Optionally, the recommendation rate scoring unit is specifically configured to obtain, based on the plurality of third session data, a first number of times that each broker in the plurality of brokers recommends the object for the preset user, and a first number of replies of the preset user to each broker; determining a plurality of recommendation rates based on the plurality of first times and the plurality of first reply times; and determining the recommendation rate score corresponding to the preset broker based on the ranking of the recommendation rates corresponding to the preset broker in the plurality of recommendation rates.

Optionally, the index scoring unit is specifically configured to determine a plurality of ranking evaluation index scores corresponding to the plurality of brokers based on the recommendation tag orders corresponding to the plurality of brokers in the plurality of third session data; wherein each of the brokers corresponds to one of the rank evaluation index scores; and determining the label recommendation score corresponding to the preset broker based on the ranking assessment index score corresponding to the preset broker.

Optionally, the apparatus further comprises:

and the prompt module is used for responding to at least one of the first evaluation score, the second evaluation score and the third evaluation score to meet a preset condition and prompting at least one prompt message for the preset broker.

According to still another aspect of the embodiments of the present disclosure, there is provided an electronic device including:

a memory for storing a computer program product;

a processor configured to execute the computer program product stored in the memory, and when executed, implement the method of evaluating a broker as described in any one of the embodiments above.

According to a further aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method of evaluating a broker according to any of the embodiments described above.

According to a further aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer program instructions which, when executed by a processor, implement the method of brokerage evaluation of any of the embodiments described above.

Based on the method and the device for evaluating the brokers, the electronic equipment and the storage medium provided by the embodiments of the present disclosure, a plurality of session data of interaction between at least one broker and at least one user for at least one object in a set duration are obtained, and a session set is obtained; wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects; determining a first evaluation score of a preset broker based on a plurality of first session data corresponding to the preset broker in the session set; determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set; wherein the plurality of second session data corresponds to a plurality of the brokers; determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to a preset user in the session set; wherein the plurality of third session data corresponds to a plurality of the brokers; determining a capacity of the preset broker based on at least one of the first, second, and third evaluation scores; according to the method and the device, based on session data corresponding to the same broker, the same object and the same user of the object, the capacities of the preset broker in different dimensions are respectively determined, comprehensive evaluation of the broker is achieved, and the problem that the prior art is based on manual evaluation is solved.

The technical scheme of the present disclosure is described in further detail below through the accompanying drawings and examples.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing embodiments thereof in more detail with reference to the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the disclosure, and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure, without limitation to the disclosure. In the drawings, like reference numerals generally refer to like parts or steps.

FIG. 1 is a flow chart of a method of broker evaluation provided in an exemplary embodiment of the present disclosure;

FIG. 2 is a schematic flow chart of step 104 in the embodiment of FIG. 1 of the present disclosure;

FIG. 3 is a schematic flow chart of step 1044 in the embodiment of FIG. 2 of the present disclosure;

FIG. 4 is a schematic flow chart of step 106 in the embodiment of FIG. 1 of the present disclosure;

FIG. 5 is a schematic flow chart of step 108 in the embodiment of FIG. 1 of the present disclosure;

FIG. 6 is a schematic diagram of a broker's evaluation device provided in an exemplary embodiment of the present disclosure;

fig. 7 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present disclosure and not all of the embodiments of the present disclosure, and that the present disclosure is not limited by the example embodiments described herein.

It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless it is specifically stated otherwise.

It will be appreciated by those of skill in the art that the terms "first," "second," etc. in embodiments of the present disclosure are used merely to distinguish between different steps, devices or modules, etc., and do not represent any particular technical meaning nor necessarily logical order between them.

It should also be understood that in embodiments of the present disclosure, "plurality" may refer to two or more, and "at least one" may refer to one, two or more.

It should also be appreciated that any component, data, or structure referred to in the presently disclosed embodiments may be generally understood as one or more without explicit limitation or the contrary in the context.

In addition, the term "and/or" in this disclosure is merely an association relationship describing an association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the front and rear association objects are an or relationship. The data referred to in this disclosure may include unstructured data, such as text, images, video, and the like, as well as structured data.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and that the same or similar features may be referred to each other, and for brevity, will not be described in detail.

Meanwhile, it should be understood that the sizes of the respective parts shown in the drawings are not drawn in actual scale for convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.

Embodiments of the present disclosure may be applicable to electronic devices such as terminal devices, computer systems, servers, etc., which may operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with the terminal device, computer system, server, or other electronic device include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments that include any of the above systems, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform particular tasks or implement particular abstract data types. The computer system/server may be implemented in a distributed cloud computing environment in which tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computing system storage media including memory storage devices.

Exemplary method

Fig. 1 is a flow chart of a method of evaluating a broker provided in an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device, as shown in fig. 1, and includes the following steps:

step 102, obtaining a plurality of session data of interaction between at least one broker and at least one user for at least one object within a set duration, and obtaining a session set.

Wherein each session data is determined based on interactions of one broker with one user for at least one object.

Optionally, the object may be a product object such as a house source, a vehicle, etc., the broker is a server providing object information for a user, the user obtains information of at least one object through interaction with the broker to determine whether to perform an operation (e.g. purchase, etc.) on a certain object, in this embodiment, session data is obtained through interaction between the broker and the user, each session data includes at least one session sentence of the user and at least one sentence of the broker, and when a time interval between two sentences reaches a preset time interval, all sentences before the time interval may be determined as one session data; the set duration in this embodiment may determine a specific length according to an actual application scenario, for example, 1 month, 6 months, 1 year, etc.

Step 104, determining a first evaluation score of the preset broker based on a plurality of first session data corresponding to the preset broker in the session set.

In one embodiment, a base capacity of the preset broker may be determined based on the preset broker corresponding to the plurality of first session data.

Step 106, determining a second evaluation score of the preset broker based on the plurality of second session data corresponding to the preset object in the session set.

Wherein the plurality of second session data corresponds to a plurality of brokers.

Optionally, based on the second session data corresponding to the same object, professional answering ability of the preset broker can be evaluated, and due to the fact that the plurality of brokers are corresponding, transverse comparison of the preset broker can be achieved, and the problem that evaluation is not objective only due to the session data of the preset broker is solved.

Step 108, determining a third evaluation score of the preset broker based on the plurality of third session data corresponding to the preset user in the session set.

Wherein the plurality of third session data corresponds to a plurality of brokers.

Optionally, based on third session data corresponding to the same user, the requirement mining capability of the preset broker can be evaluated, and due to the fact that the plurality of brokers are corresponding, transverse comparison of the preset broker can be achieved, and the problem that evaluation is not objective only due to session data of the preset broker is solved.

Step 110, determining a capacity of the preset broker based on at least one of the first, second, and third rating scores.

In this embodiment, a first evaluation score, a second evaluation score and a third evaluation score are determined for a preset broker respectively, and when the preset broker is actually checked, at least one of the first evaluation score, the second evaluation score and the third evaluation score can be selected to determine the capabilities of the preset broker in different aspects, for example, the professional answering capability of the broker is determined through the second evaluation value, and when the preset broker is required to promote a specific capability, only one or more evaluation scores corresponding to the preset broker can be displayed, so that the concentration degree of the specific capability is improved, and the problem of distraction caused by displaying all the evaluation scores is avoided.

According to the method for evaluating the brokers, which is provided by the embodiment of the disclosure, a plurality of session data of interaction between at least one broker and at least one user for at least one object in a set duration are obtained, and a session set is obtained; wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects; determining a first evaluation score of a preset broker based on a plurality of first session data corresponding to the preset broker in the session set; determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set; wherein the plurality of second session data corresponds to a plurality of the brokers; determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to a preset user in the session set; wherein the plurality of third session data corresponds to a plurality of the brokers; determining a capacity of the preset broker based on at least one of the first, second, and third evaluation scores; according to the method and the device, based on session data corresponding to the same broker, the same object and the same user of the object, the capacities of the preset broker in different dimensions are respectively determined, comprehensive evaluation of the broker is achieved, and the problem that the prior art is based on manual evaluation is solved.

As shown in fig. 2, step 104 may include the following steps, based on the embodiment shown in fig. 1, described above:

step 1041, determining at least one conversation sentence of the plurality of conversation data corresponding to the preset broker.

In this embodiment, session data in the session set is screened from the dimension of the broker, and all session data corresponding to the preset broker are determined, where the session data is interaction data between the broker and the user, so that session sentences of the broker are included, and also session sentences of the user are included, and optionally, all session sentences output by the preset broker in the interaction process are obtained through screening.

Step 1042, determining a first base score for the preset broker based on a number of first preset words and second preset words included in the at least one conversational sentence.

Optionally, the first preset word and the second preset word may be a start word and an end word, and the number of the first preset word and the second preset word included in the conversation sentence may be determined by a natural language recognition technology, for example, the first preset word is set to be words such as "hello", "your good", and the second preset word is set to be words such as "bye", "rebecce", and the like; the first base score may be determined directly by the number of first and second preset words, or may be determined based on a ratio of the number of first and second preset words to the number of all conversational sentences corresponding to the preset broker.

Step 1043, determining an expression score of the preset broker based on a number of third preset words included in the at least one conversational sentence.

Optionally, the third preset words may be words with harder expression, the number of the third preset words included in the conversation sentence may be determined through a natural language recognition technology, and the third preset words may be set according to an application scenario; alternatively, the expression score may be directly determined by recognizing the number of the obtained third preset words, or the expression score may be determined based on a ratio of the number of the third preset words to the number of all conversational sentences corresponding to the preset broker.

Step 1044, determining a first valuation score for the preset broker based on the first base score and the expression score.

According to the method, the device and the system, the conversation sentences output by the preset broker in the plurality of conversation data are used as the basis, the evaluation of the basic capacity of the preset broker is achieved, and because the plurality of conversation data correspond to the preset time, compared with the prior art that the broker is scored based on a single conversation, the objectivity of the first evaluation score is improved through more conversation sentences obtained in the preset time.

As shown in fig. 3, step 1044 may include the following steps, based on the embodiment shown in fig. 2, described above:

Step 301, determining a performance change rate of a preset broker, and determining a ranking of the performance change rates of the preset broker in a set of brokers to which the preset broker belongs.

The broker set includes a plurality of brokers; alternatively, the set of brokers may be all brokers of the same store, etc.; the performance change rate may be determined based on a difference between average performance over different time periods, for example, in months, and the performance change rate b may be determined based on the following formula (1):

wherein i represents the i-th month; j represents month j; j-i>1；m _j Mean performance for month j; m is m _i Mean performance for month i; n is the month number corresponding to the preset duration, for example, the available value is 6, that is, the session set includes session data within half a year; r is a first preset value, and the value can be set according to an actual application scene.

Based on the above equation (1), a performance change rate for each broker in the set of brokers may be determined.

Step 302, determining a second base score based on the performance change rate, the ranking of the performance change rates, and the first base score.

In this embodiment, after determining the rate of change of performance of each broker in the set of brokers, i.e.The rate of change of performance ordering of the preset brokers in the broker collection may be determined, alternatively, a second base score base may be determined based on the following equation (2) ₂ ：

Wherein, base ₁ Representing a first base score; rank represents a ranking of performance change rates corresponding to a preset broker; lambda is a second preset value, and the value can be set according to the actual application scene.

Step 303, determining a first evaluation score of the preset broker based on the second base score and the expression score.

Alternatively, the second base score and the expression score may be weighted and summed to obtain a first evaluation score, e.g., the first evaluation score may be determined by the following equation (3) ₁ ：

score ₁ ＝w ₁ *base ₂ +w ₂ *base ₃ Formula (3)

Wherein w is ₁ And w ₂ Representing a first preset weight value and a second preset weight value which respectively correspond to the second basic score and the expression score, wherein the values can be set according to experience values or specific scenes; base ₂ Representing a second base score; base ₃ Representing the expression score; in this embodiment, the weighted summation integrates a plurality of pieces of information in the session data, so that accuracy of the first evaluation score is improved, and in combination with ranking of the performance change rates, brokers with later ranks (for example, 20% after ranking) can be screened out, poor basic capabilities of the brokers are determined, and prompt and other treatments can be performed on the brokers.

Based on the above embodiment, step 301 may further include:

The performance change rate of the preset broker is determined based on the average performance of the preset broker over each sub-preset duration over the preset duration.

The preset time length comprises a plurality of sub-preset time lengths; for example, the preset time period is half a year, the sub-preset time period is 1 month, and the preset time period includes 6 sub-preset time periods.

Alternatively, the performance change rate for each broker may be determined based on equation (1) above.

A ranking of the performance change rates corresponding to the preset brokers is determined based on a plurality of performance change rates corresponding to a plurality of brokers included in the set of brokers to which the preset broker belongs.

In this embodiment, after determining the performance change rate corresponding to each broker in the broker set, the ranking of the performance change rates may be obtained only by ranking according to size alignment, and the ranking mode may be from large to small or from small to large, and only the same ranking mode is required to be maintained for all preset brokers.

As shown in fig. 4, step 106 may include the following steps, based on the embodiment shown in fig. 1, described above:

step 1061, determining a adoption score corresponding to the preset broker based on the number of times the preset broker does not adopt the recommended sentences in the plurality of second session data.

In this embodiment, during the session between the broker and the user, the session assistant may provide the recommended sentence in a prompting manner (for example, pop up in a prompting card manner, only for viewing by the broker), where the broker may or may not adopt the recommended sentence, and the adoption is to use the recommended sentence for interaction with the user; alternatively, the unadopted number may be all the numbers of times that the preset broker does not adopt the recommended sentences, or an intersection of all the recommended sentences that the preset broker does not adopt with all the recommended sentences that all the brokers corresponding to the plurality of second conversation data adopt, with the number of recommended sentences included in the intersection as the unadopted number.

Step 1062, determining a response rate score corresponding to the preset broker based on the explanation response rate of the preset broker to the preset object in the plurality of second session data.

In this embodiment, since all the second session data correspond to the same preset object, the explanation reply rate may be determined by a ratio between the number of times the preset broker explains the preset object and the number of times the user replies.

Step 1063, determining a solution rate score corresponding to the preset broker based on the solution rates of the preset broker in the plurality of second session data.

Optionally, for a user's question, whether the broker answers, may evaluate whether the broker reaction is positive; the embodiment determines the answer rate by presetting a ratio between the answer number of the broker to the questions of the users and the question number corresponding to the questions.

Step 1064, determining a second valuation score for the preset broker based on the adoption score, the return score, and the answer score.

In the embodiment, the adoption score, the response rate score and the answer rate score are combined to determine the second evaluation score of the preset broker, so that the familiarity of the preset broker to the preset floor and the enthusiasm of interaction with the user are reflected; alternatively, the second evaluation score may be determined by means of weighted summation, e.g., based on the following equation (4) ₂ ：

score ₂ ＝base ₀ (w ₃ +w ₄ +w ₅ )+base ₄ *w ₃ +base ₅ *w ₄ +base ₆ *w ₅

Formula (4)

Wherein, base ₀ Representing a basic score which is used for avoiding that the second evaluation score is 0 or too small, wherein the value can be set according to actual conditions; w (w) ₃ 、w ₄ 、w ₅ The weight values corresponding to the adoption scores, the response rate scores and the answer rate scores are respectively represented, and the values can be set according to experience values or specific scenes; base ₄ 、base ₅ 、base ₆ Respectively representing the adoption score, the response rate score and the answer rate score.

Optionally, on the basis of the above embodiment, step 1061 may further include:

obtaining n recommended sentences which are not adopted by the preset brokers in the plurality of second conversation data, and a recommended sentence subset which is adopted by the plurality of brokers in the plurality of second conversation data.

Wherein n is equal to or greater than zero;

determining the unadopted times of a preset broker based on intersections between the n recommended sentences and the recommended sentence subsets;

and determining the adoption scores corresponding to the preset brokers based on the unadopted times.

In this embodiment, in the session data corresponding to the same object, when the same recommended sentences exist in different session data, if recommended sentences are adopted in other session data (other brokers), but the recommended sentences are not adopted by the broker (preset broker) in the current session, diagnosis is required, and as an evaluation criterion for the preset broker, the number of unapplication times is determined by the intersection between n recommended sentences and the subset of recommended sentences, and based on the number of unapplication times, the adoption scoring base can be determined by the following formula (5) ₄ ：

base ₄ ＝base ₀ -n 5 formula (5)

Wherein, base ₀ Representing basic scores, wherein the values can be set according to actual conditions; n represents the unadopted number of times.

Optionally, on the basis of the above embodiment, step 1062 includes:

and determining the explanation times of the plurality of brokers to the preset object in the plurality of second session data and the replying times of the explanation, and determining a plurality of explanation replying rates corresponding to the plurality of brokers.

Wherein each broker corresponds to an explanation return rate.

And determining a response rate score corresponding to the preset broker based on the ranking of the explanation response rates corresponding to the preset broker in the plurality of explanation response rates.

In this embodiment, the explanation degree of each broker on the preset object is expressed by the explanation reply rate, so as to reflect the familiarity of the broker on the preset object, and reflect the capability of the preset broker at the angle; after determining the explanation replies for each broker, the brokers are ranked by size (e.g., from small to large,or from large to small, etc.), a quantile corresponding to the explanation response rate corresponding to the preset broker is determined based on the ranking by means of a quantile solution, and a response rate score is determined based on the quantile, e.g., a response rate score base is determined based on the following equation (6) ₅ ：

base ₅ ＝base ₀ +f ₁ *(100-base ₀ ) Formula (6)

Wherein, base ₀ Representing basic scores, wherein the values can be set according to actual conditions; f (f) ₁ And representing the quantile corresponding to the explanation reply rate corresponding to the preset broker.

Optionally, on the basis of the above embodiment, step 1063 includes:

the method comprises the steps of determining the number of questions asked by a plurality of users and the number of questions answered by a plurality of brokers to the plurality of users in a plurality of second session data, and determining a plurality of answering rates corresponding to the plurality of brokers.

Wherein each broker corresponds to a solution rate.

And determining a solution rate score corresponding to the preset broker based on the ranking of the solution rates corresponding to the preset broker in the plurality of solution rates.

In this embodiment, the solution rate of a broker is determined by the ratio of the number of times that the broker answers questions of a plurality of users to the number of times that the broker asks questions of a plurality of users, for example, two users ask a broker a 5 times respectively, and the broker a solves for two users 3 times and 4 times respectively, then the solution rate obtained is (3+4)/(5+5) =0.7; ranking (e.g., from small to large, or from large to small, etc.) based on the solutions corresponding to each broker, determining a quantile corresponding to the solutions corresponding to the preset broker by quantile solving based on the ranking, and determining a solution score based on the quantile, e.g., determining a solution score base based on the following equation (7) ₆ ：

base ₆ ＝base ₀ +f ₂ *(100-base ₀ ) Formula (7)

Wherein, base ₀ Represents a basal score having the meaning as in the above formula (4)The same; f (f) ₂ And representing the quantile corresponding to the explanation reply rate corresponding to the preset broker.

As shown in fig. 5, step 108 may include the following steps, based on the embodiment shown in fig. 1, described above:

step 1081, determining a recommendation rate score corresponding to the preset broker based on the recommendation rates corresponding to the preset broker in the plurality of third session data.

In this embodiment, in the interaction process between the broker and the same preset user, a plurality of objects may be recommended for the preset user, but the preset user usually replies only to the object of interest, but does not reply to the object of no interest; the degree of knowledge of the preset broker about the preset user can be evaluated through the recommendation rate.

Step 1082, determining a label recommendation score corresponding to the preset broker based on the recommended label orders of the corresponding plurality of brokers in the plurality of third session data.

Wherein each broker corresponds to a rank evaluation index score.

In this embodiment, during the interaction between the broker and the preset user, the interaction assistant may generate tag cards for the preset user in real time according to the interaction content, where the tag cards indicate preferential points of interest of the user, and because the points of interest of the object are limited, the number of tag cards is limited, enumeration may be performed, and optionally, an identification number is allocated to each different tag card to distinguish the tag cards; determining the recommended label sequence through the sequence of a plurality of label cards corresponding to the preset user, which is provided for one broker in the conversation process, so as to obtain the label recommendation score corresponding to each broker.

Step 1083, determining a third valuation score for the preset broker based on the recommendation rate score and the tag recommendation score.

In this embodiment, the recommendation rate score and the ranking evaluation index score are combined to determine the third evaluation score of the preset broker, and the points of interest of the preset user are guided out in the interaction process by recommending the interested objects to the preset user, thereby embodying the preset broker to the preset userThe familiarity, demand mining ability, and guided business ability of (c), optionally, the third evaluation score may be determined by means of weighted summation, e.g., the third evaluation score may be determined based on the following equation (8) ₃ ：

score ₃ ＝base ₀ (w ₆ +w ₇ )+base ₇ *w ₆ +base ₈ *w ₇ Formula (8)

Wherein, base ₀ Representing a basic score which is used for avoiding that the third evaluation score is 0 or too small, wherein the value can be set according to actual conditions; w (w) ₆ And w ₇ The weight values corresponding to the recommendation rate scores and the ranking evaluation index scores are respectively represented, and the values can be set according to experience values or specific scenes; base ₇ And base ₈ Respectively representing a recommendation rate score and a ranking assessment index score.

Optionally, on the basis of the above embodiment, step 1081 may further include:

based on the third session data, a first number of times that each of the plurality of brokers recommended the object to the user is preset for each broker is obtained, and a first number of replies that the preset user replies to each broker.

A plurality of recommendation rates is determined based on the plurality of first times and the plurality of first replies.

Optionally, based on the ratio of the number of replies of the preset user to the number of recommended objects of one broker, the recommendation rate corresponding to the broker is determined, for example, broker B recommends 10 times of objects (specifically, corresponding to a plurality of objects, there may be a case that one object recommends multiple times), and the preset user replies 5 times, at this time, the recommendation rate corresponding to broker B is 0.5.

Determining a recommendation rate score corresponding to the preset broker based on a ranking of the recommendation rates corresponding to the preset broker among the plurality of recommendation rates.

In this embodiment, the degree of knowledge of the preset user by each broker is indicated by the recommendation rate, i.e. the objects that are more interested (more likely to reply) by the preset user are recommended, so that the higher the recommendation rate, the higher the broker's knowledge of the preset userThe higher the solution degree is, namely, the recommendation rate score reflects the analysis capability of the broker to the user; after the recommendation rate of each broker is determined, sorting by size (e.g., from small to large, or from large to small, etc.), determining a quantile corresponding to the recommendation rate corresponding to the preset broker by quantile solving based on the sorting, and determining a recommendation rate score based on the quantile, e.g., determining a recommendation rate score base based on the following equation (9) ₇ ：

base ₇ ＝base ₀ +f ₃ * (100-base ₀ ) Formula (9)

Wherein, base ₀ Representing basic scores, wherein the values can be set according to actual conditions; f (f) ₃ Representing the quantile corresponding to the recommendation rate corresponding to the preset broker.

Optionally, on the basis of the above embodiment, step 1082 may further include:

a plurality of ranking assessment indicator scores corresponding to the plurality of brokers are determined based on the recommended tag orders corresponding to the plurality of brokers in the plurality of third session data.

And determining a label recommendation score corresponding to the preset broker based on the ranking evaluation index score corresponding to the preset broker.

In this embodiment, NDCG (Normalized Discounted Cumulative Gain, normalized damage cumulative gain) scores of each broker for tag cards of preset users are calculated to obtain ranking evaluation index scores; this embodiment laterally compares sessions corresponding to the same user, with the content of the comparison being NDCG scores corresponding to each broker-corresponding session, based on which a tag recommendation score may be determined, e.g., a tag recommendation score base may be determined based on the following equation (10) ₈ ：

base ₈ ＝base ₀ +n3*(100-base ₀ ) Formula (10)

Wherein, base ₀ Representing basic scores, wherein the values can be set according to actual conditions; n3 represents NDCG score corresponding to the preset broker.

In some optional embodiments, the method provided in this embodiment may further include:

at least one hint is presented to the pre-set broker in response to at least one of the first, second, and third rating values meeting a pre-set condition.

In this embodiment, the first evaluation score, the second evaluation score and the third evaluation score are determined for the preset broker through the calculation in the foregoing embodiment, and the three evaluation scores respectively evaluate the broker from different angles, when different abilities of the preset broker need to be processed, one or more evaluation scores may be displayed, and corresponding prompt information is given to the preset broker based on the relationship between the displayed evaluation score and other brokers or the relationship between the displayed evaluation score and the preset average score, so as to promote the ability of the preset broker corresponding to the evaluation score and promote the growth of the preset broker.

It should be understood that the base involved in various formulas in the various embodiments of the present disclosure described above ₀ The values of (a) are the same in the same formula, and may be the same or different in different formulas.

Any of the broker evaluation methods provided by the embodiments of the present disclosure may be performed by any suitable device having data processing capabilities, including, but not limited to: terminal equipment, servers, etc. Alternatively, any of the broker's evaluation methods provided by the embodiments of the present disclosure may be executed by a processor, such as the processor executing any of the broker's evaluation methods mentioned by the embodiments of the present disclosure by invoking corresponding instructions stored in a memory. And will not be described in detail below.

Exemplary apparatus

Fig. 6 is a flow chart of a method of evaluating a broker provided in an exemplary embodiment of the present disclosure. The apparatus as shown in fig. 6 includes:

the data obtaining module 61 is configured to obtain a plurality of session data of interactions between at least one broker and at least one user for at least one object within a set duration, so as to obtain a session set.

Wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects.

A first evaluation module 62 is configured to determine a first evaluation score for a preset broker based on a plurality of first session data for the preset broker in the session set.

A second evaluation module 63, configured to determine a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set.

Wherein the plurality of second session data corresponds to a plurality of the brokers.

A third evaluation module 64 is configured to determine a third evaluation score of the preset broker based on a plurality of third session data corresponding to the preset user in the session set.

Wherein the plurality of third session data corresponds to a plurality of the brokers.

A capacity determination module 65 for determining a capacity of the preset broker based on at least one of the first, second, and third evaluation scores.

According to the evaluation device for the brokers, provided by the embodiment of the disclosure, a plurality of session data of interaction between at least one broker and at least one user for at least one object in a set duration are obtained, and a session set is obtained; wherein each of the session data is determined based on interactions of one of the brokers with one of the users for at least one of the objects; determining a first evaluation score of a preset broker based on a plurality of first session data corresponding to the preset broker in the session set; determining a second evaluation score of the preset broker based on a plurality of second session data corresponding to a preset object in the session set; wherein the plurality of second session data corresponds to a plurality of the brokers; determining a third evaluation score of the preset broker based on a plurality of third session data corresponding to a preset user in the session set; wherein the plurality of third session data corresponds to a plurality of the brokers; determining a capacity of the preset broker based on at least one of the first, second, and third evaluation scores; according to the method and the device, based on session data corresponding to the same broker, the same object and the same user of the object, the capacities of the preset broker in different dimensions are respectively determined, comprehensive evaluation of the broker is achieved, and the problem that the prior art is based on manual evaluation is solved.

Optionally, the first evaluation module 62 includes:

Optionally, the second evaluation module 63 includes:

Optionally, the third evaluation module 64 includes:

an index scoring unit, configured to determine a tag recommendation score corresponding to the preset broker based on a recommended tag order corresponding to the plurality of brokers in the plurality of third session data; wherein each of the brokers corresponds to one of the rank evaluation index scores;

Optionally, the apparatus further comprises:

Exemplary electronic device

Next, an electronic device according to an embodiment of the present disclosure is described with reference to fig. 7. The electronic device may be either or both of the first device 100 and the second device 200, or a stand-alone device independent thereof, which may communicate with the first device and the second device to receive the acquired input signals therefrom.

Fig. 7 illustrates a block diagram of an electronic device according to an embodiment of the disclosure.

As shown in fig. 7, the electronic device 70 includes one or more processors 71 and memory 72.

The processor 71 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 70 to perform desired functions.

The memory may store one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or nonvolatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program products may be stored on the computer readable storage medium that can be run by a processor to implement the broker evaluation methods and/or other desired functions of the various embodiments of the present disclosure described above.

In one example, the electronic device 70 may further include: an input device 73 and an output device 74, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).

For example, when the electronic device is the first device 100 or the second device 200, the input means 73 may be a microphone or a microphone array as described above for capturing an input signal of a sound source. When the electronic device is a stand-alone device, the input means 73 may be a communication network connector for receiving the acquired input signals from the first device 100 and the second device 200.

In addition, the input device 73 may also include, for example, a keyboard, a mouse, and the like.

The output device 74 may output various information to the outside, including the determined distance information, direction information, and the like. The output device 74 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, only some of the components of the electronic device 70 that are relevant to the present disclosure are shown in fig. 7 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, the electronic device 70 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer readable storage Medium

In addition to the methods and apparatus described above, embodiments of the present disclosure may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in a method of evaluating a broker according to various embodiments of the present disclosure described in the above section of the present description.

The computer program product may write program code for performing the operations of embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium, having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a method of evaluating a broker according to various embodiments of the present disclosure described in the above "exemplary methods" section of the present description.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The basic principles of the present disclosure have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present disclosure. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, since the disclosure is not necessarily limited to practice with the specific details described.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different manner from other embodiments, so that the same or similar parts between the embodiments are mutually referred to. For system embodiments, the description is relatively simple as it essentially corresponds to method embodiments, and reference should be made to the description of method embodiments for relevant points.

The block diagrams of the devices, apparatuses, devices, systems referred to in this disclosure are merely illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above-described sequence of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the sequence specifically described above unless specifically stated otherwise. Furthermore, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the apparatus, devices and methods of the present disclosure, components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered equivalent to the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the disclosure to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. A method of evaluating a broker, comprising:

2. The method of claim 1, wherein the determining a first valuation score for the preset broker based on a plurality of first session data for the corresponding preset broker in the session set comprises:

3. The method of claim 2, wherein the determining the first valuation score for the preset broker based on the first base score and the expression score comprises:

4. A method according to claim 3, wherein the determining the rate of change of performance of the preset broker and determining a corresponding ranking of the rate of change of performance of the preset broker in a set of brokers to which the preset broker belongs comprises:

5. The method of any of claims 1-4, wherein determining a second valuation score for the preset broker based on a plurality of second session data for a corresponding preset object in the session set comprises:

6. The method of claim 5, wherein the determining the adoption score corresponding to the preset broker based on the number of times the preset broker does not take a recommended sentence in the plurality of second session data comprises:

7. The method of claim 5, wherein the determining a response rate score corresponding to the preset broker based on the explanation response rate of the preset broker to the preset object in the plurality of second session data comprises:

8. The method of claim 5, wherein the determining a solution rate score for the preset broker based on the solution rates for the preset brokers in the plurality of second session data comprises:

9. The method of any of claims 1-4, wherein determining a third valuation score for the preset broker based on a plurality of third session data for a corresponding preset user in the session set comprises:

10. The method of claim 9, wherein the determining a recommendation rate score for the preset broker based on a recommendation rate for the preset broker in the plurality of third session data comprises:

11. The method of claim 9, wherein the determining the tag recommendation score corresponding to the preset broker based on the recommended tag orders corresponding to the plurality of brokers in the plurality of third session data comprises:

12. The method of any one of claims 1-4, further comprising:

13. A broker evaluation device, comprising:

14. An electronic device, comprising:

a memory for storing a computer program product;

a processor for executing a computer program product stored in the memory, which when executed, implements a method of brokerage evaluation as claimed in any one of the preceding claims 1-12.

15. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement a method of evaluating a broker as claimed in any one of claims 1 to 12.

16. A computer program product comprising computer program instructions which, when executed by a processor, implement a method of brokerage evaluation as claimed in any one of claims 1 to 12.