CN115328745A

CN115328745A - Deployment effect evaluation method and device, electronic equipment and readable storage medium

Info

Publication number: CN115328745A
Application number: CN202210929453.7A
Authority: CN
Inventors: 赵雅琼
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2022-08-03
Filing date: 2022-08-03
Publication date: 2022-11-11

Abstract

The disclosure provides a deployment effect evaluation method and device, electronic equipment and a readable storage medium, and relates to the technical field of computers, in particular to the technical field of continuous deployment. The specific implementation scheme is as follows: acquiring change related information corresponding to current code change operation of a code module to be evaluated in a current deployment stage, and/or acquiring current alarm information of the code module to be evaluated in the current deployment stage, wherein the code module to be evaluated is a code module of which the current alarm information meets preset alarm conditions; and determining the deployment effect of the code module to be evaluated based on the change related information and/or the current alarm information. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is favorably reduced, and the processing efficiency is improved.

Description

Deployment effect evaluation method and device, electronic equipment and readable storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a deployment effect evaluation method, apparatus, electronic device, and readable storage medium.

Background

The continuous deployment is a process of automatically deploying the code after the integration test to a production environment on the basis of a continuous integration technology, generally, the deployment process is divided into a plurality of deployment phases in the continuous deployment, and after the deployment of each deployment phase is completed, the deployment effect is verified.

In the existing scheme, the deployment effect generally needs to be judged by manual assistance.

Disclosure of Invention

In order to solve at least one of the above-mentioned drawbacks, the present disclosure provides a deployment effectiveness evaluation method, apparatus, electronic device, and readable storage medium.

According to a first aspect of the present disclosure, there is provided a deployment effectiveness evaluation method, the method comprising:

acquiring change related information corresponding to current code change operation of a code module to be evaluated in a current deployment stage, and/or acquiring current alarm information of the code module to be evaluated in the current deployment stage, wherein the code module to be evaluated is a code module of which the current alarm information meets preset alarm conditions;

and determining the deployment effect of the code module to be evaluated based on the change related information and/or the current alarm information.

According to a second aspect of the present disclosure, there is provided a deployed effect assessment apparatus, the apparatus comprising:

the information acquisition module is used for acquiring change related information corresponding to current code change operation performed by the code module to be evaluated in a current deployment stage, and/or current alarm information of the code module to be evaluated in the current deployment stage, wherein the code module to be evaluated is a code module of which the current alarm information meets preset alarm conditions;

and the deployment effect evaluation module is used for determining the deployment effect of the code module to be evaluated based on the change related information and/or the current alarm information.

According to a third aspect of the present disclosure, there is provided an electronic apparatus including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the deployment effectiveness assessment method.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the deployment effectiveness evaluation method described above.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the deployment effectiveness assessment method described above.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flow chart of a deployment effectiveness evaluation method according to an embodiment of the present disclosure;

FIG. 2 is an architecture diagram of a deployment effectiveness evaluation system provided by an embodiment of the present disclosure;

fig. 3 and fig. 4 are trend graphs of changes of relevant indexes in the operation process of the deployment effectiveness evaluation system provided by the embodiment of the disclosure;

fig. 5 is a schematic structural diagram of a deployment effect evaluation apparatus according to an embodiment of the present disclosure;

fig. 6 is a block diagram of an electronic device for implementing the deployment effectiveness evaluation method according to the embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of embodiments of the present disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the related art, after deployment of a code module is completed, the deployed code module generally needs to be tested (for example, an a/B test is adopted), and a developer performs auxiliary judgment on a deployment effect according to a test result, so that the method has high dependence degree on manpower and low processing efficiency.

The deployment effect evaluation method, device, electronic device and readable storage medium provided by the embodiments of the present disclosure aim to solve at least one of the above technical problems in the prior art.

Fig. 1 shows a schematic flow chart of a deployment effect evaluation method provided by an embodiment of the present disclosure, as shown in fig. 1, the method may mainly include:

step S110: and obtaining change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage, and/or obtaining current alarm information of the code module to be evaluated in the current deployment stage, wherein the code module to be evaluated is a code module of which the current alarm information meets preset alarm conditions.

Step S120: and determining the deployment effect of the code module to be evaluated based on the change related information and/or the current alarm information.

In the continuous deployment, the deployment process is generally divided into a plurality of deployment phases, and after the deployment of each deployment phase is completed, the deployment effect is verified.

And code changing operation, namely deploying the changed code. Code change operations may include change operations for code such as programs, configuration information, and vocabularies.

After the code change operation of the code module in the current deployment stage is completed, each changed code module can be tested, and a test result is obtained. Alarm information generated by the code module in the test process can be included in the test result.

For convenience of description, in the following description, a code change operation performed by a code module in a current deployment stage may be referred to as a current code change operation, and alarm information generated by the code module in the current deployment stage may be referred to as current alarm information.

In the embodiment of the disclosure, the current alarm information can reflect the situation that the deployment effect of the code module is not good in the current deployment stage to a certain extent, the alarm condition can be set, the current alarm information of the code module is counted to obtain the statistical result of the alarm information, when the statistical result of the alarm information meets the alarm condition, the situation that the deployment effect of the code module is not good in the current deployment stage is indicated, and the code module can be used as a module to be evaluated to evaluate the deployment effect subsequently.

As an example, the alarm information may include an alarm log, when a log at a fatal error (false) level in the alarm log indicates that a serious error may occur in the code module, which may result in poor deployment of the code module, and the alarm condition may be set such that the alarm information includes a log at a false level.

In the embodiment of the disclosure, code modules with current alarm information meeting alarm conditions can be screened out from all code modules with code change operation in the current deployment stage, that is, code modules with poor deployment effect possibly existing in the current deployment stage are screened out and used as modules to be evaluated, and the deployment effect of the modules to be evaluated in the current deployment stage is evaluated.

In the embodiment of the disclosure, the deployment effect can reflect the operation effect of the code module after the code change operation of the current deployment stage is completed.

In the embodiment of the disclosure, the change related information is information of the module to be evaluated, which is related to the code change operation. The specific situation of code change in the module to be evaluated can be obtained through changing the related information, and the specific situation of alarm can be obtained through the current alarm information, so that the deployment effect of the code module to be evaluated can be accurately and effectively evaluated based on the change related information and/or the current alarm information.

According to the method provided by the embodiment of the disclosure, the deployment effect of the code module to be evaluated is determined by determining the code module to be evaluated of which the current alarm information meets the preset alarm condition, based on the change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage, and/or the current alarm information of the code module to be evaluated in the current deployment stage. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is favorably reduced, and the processing efficiency is improved.

In the embodiment of the present disclosure, if the deployment effect of the code module to be evaluated is evaluated directly by a manual evaluation method, since the number of the code modules to be evaluated is generally large, much labor consumption is caused. The accuracy of the code modules to be evaluated can be improved by improving the level of the alarm condition, and the number of the code modules to be evaluated can be reduced to improve the processing efficiency, but the mode may bring the reduction of the recall rate. In the scheme, the deployment effect of the module to be evaluated is automatically and effectively evaluated, so that the effect of improving the processing efficiency is realized.

As an example, the deployment effect may include a better deployment effect and a poor deployment effect, when the deployment effect is the better deployment effect, it indicates that the running effect of the code module to be evaluated is good, and the deployment in the next deployment stage may be performed, and when the deployment effect is the poor deployment effect, it indicates that the running effect of the code module to be evaluated is not good, and at this time, the code module in which the code change operation has occurred in the current deployment stage is rolled back.

In the embodiment of the disclosure, the current alarm information may be obtained by crawling an alarm log to a device in which a code module to be evaluated is deployed. The method has the advantages that the use range is wide, when the central control system is consistent with a platform of a system deployed by a code module to be evaluated, the central control system can log in a system crawling alarm log deployed by the code module to be evaluated through an application programming Interface (api) of the central control system, but the method has the defect of being seriously dependent on the stability of the api, and when the system deployed by the code module to be evaluated has a large load, the system is logged in at a very high failure rate.

In the embodiment of the present disclosure, the current alarm information may also be obtained by subscribing to an alarm log reported by the code module to be evaluated. The method can subscribe the log related to the code module to be evaluated in real time on line, for example, a real-time stream data subscription such as a siren (bigpipe) can be adopted, so that the timeliness is high, and the alarm information can be quickly acquired after the alarm is generated.

In actual execution, the two modes can be matched to obtain alarm information, the coverage of obtaining the alarm information is improved by subscribing an online alarm log in a code module which cannot be subjected to high-frequency change, and the alarm log is crawled to equipment with a code module to be evaluated in the rest code modules.

In an optional manner of the present disclosure, determining a deployment effect of a code module to be evaluated based on change related information and/or current alarm information includes:

matching a target evaluation strategy in preset evaluation strategies based on the change related information and/or the current alarm information, wherein the evaluation strategies are pre-configured with corresponding sub-effect evaluation indexes;

and determining an effect evaluation index of the code module to be evaluated based on a target sub-effect evaluation index corresponding to the target evaluation strategy, wherein the effect evaluation index is used for representing the deployment effect of the code module to be evaluated.

In the embodiment of the disclosure, the evaluation strategy can be formulated according to experience of manually evaluating the deployment effect in history, and the evaluation strategy can be set from various data dimensions to ensure effectiveness of the deployment effect evaluation.

The evaluation strategy can be preconfigured with corresponding sub-effect evaluation indexes, and the sub-effect evaluation indexes can represent the degree of influence of the evaluation strategy on the deployment effect. As an example, a higher value of the sub-effect evaluation index indicates a higher degree of influence thereof on the deployment effect.

The influence of the sub-effect evaluation index on the deployment effect can be positive influence or negative influence. The positive effects can cause the deployment effect to tend to be better, and the negative effects can cause the deployment effect to tend to be worse.

In the embodiment of the disclosure, when evaluating the deployment effect of a certain code module to be evaluated, only a part of evaluation strategies may be applied to the code module to be evaluated, and the part of evaluation strategies may be used as target evaluation strategies, and the effect evaluation index of the code module to be evaluated is determined according to the target sub-effect evaluation index corresponding to the target evaluation strategies.

In the embodiment of the disclosure, the effect evaluation index is used for representing the deployment effect of the code module to be evaluated.

As an example, the effect evaluation index may be a numerical value, wherein a higher numerical value indicates a better deployment effect, and a lower numerical value indicates a worse deployment effect.

As an example, each sub-effect evaluation index may be preconfigured with different weights according to the influence degree of the evaluation policy, and the target sub-effect evaluation index is weighted to obtain the effect evaluation index.

In an optional manner of the present disclosure, the evaluation policy includes at least two items, the evaluation policy corresponds to a preset matching sequence, and a target evaluation policy is matched in the preset evaluation policy based on the change related information and/or the current alarm information, including:

and sequentially matching with each preset evaluation strategy based on the change related information and/or the current alarm information according to the matching sequence until the effect evaluation index meeting the preset index condition can be determined based on the target sub-effect evaluation index corresponding to the target evaluation strategy obtained by matching.

In the embodiment of the present disclosure, a matching order corresponding to each evaluation policy may be preconfigured, where the matching order represents a sequence of matching each evaluation policy when a target evaluation policy is obtained by matching.

In the embodiment of the present disclosure, the matching sequence may be set according to the time consumed for matching each evaluation policy, the processing resource amount consumed for matching, and the degree of influence of the evaluation policy on the deployment effect.

As an example, a less time is consumed for matching, a less amount of processing resources is consumed for matching, and an evaluation policy with a higher impact on deployment effect may be configured with a more advanced matching order.

In the embodiment of the disclosure, in the process of sequentially matching the evaluation strategies according to the matching sequence, the effect evaluation index can be determined in real time according to the target sub-effect evaluation index corresponding to the matched target evaluation strategy. Specifically, the effect evaluation index may be recalculated each time one target evaluation policy is determined, i.e., according to its corresponding target sub-effect evaluation index.

In the process that the effect evaluation index is updated along with the matched target evaluation strategy, when the effect evaluation index meets the preset index condition, the deployment effect can be judged according to the matched target evaluation strategy, the evaluation strategy does not need to be matched continuously, and all the target evaluation strategies do not need to be matched. At this time, the current deployment effect evaluation flow can be ended, so that the calculation power is saved, the processing pressure is reduced, and the processing efficiency is improved.

As an example, 10 evaluation strategies are sequentially marked as 1 st, 2 nd, 8230, and 10 th according to the matching sequence of the evaluation strategies. The evaluation strategies can be sequentially matched according to the matching sequence from the evaluation strategy 1, and the effect evaluation index is recalculated according to the target sub-effect evaluation index corresponding to the matched target evaluation strategy. When the 5 th evaluation strategy is matched, the 1 st, 2 nd and 5 th evaluation strategies are determined as the target evaluation strategies, and the effect evaluation indexes calculated according to the target sub-effect evaluation indexes corresponding to the 1 st, 2 nd and 5 th evaluation strategies meet the preset index conditions, the current deployment effect evaluation process can be ended, and the matching of the 6 th to 10 th evaluation strategies is not performed.

As an example, two preset index conditions may be set, where one is that when the effect evaluation index is greater than a, it is determined that the deployment effect is better; and the other is that when the effect evaluation index is smaller than B, the deployment effect is judged to be poor.

In an optional aspect of the present disclosure, the changing the related information includes:

starting and ending time of current code change operation;

a current equipment room corresponding to the current deployment stage;

the corresponding change content of the current code change operation;

and the current equipment of the current alarm information appears.

For convenience of description, in the following description, the equipment room corresponding to the current deployment stage may be referred to as a current equipment room for short, and the equipment with the current alarm information is referred to as the current equipment.

In the embodiment of the present disclosure, the change-related information may further include, but is not limited to, an initiation platform of the code change operation; the code change type, the current deployment stage, the deployment stage before the current deployment stage, the deployment stage after the current deployment stage, and the like, so as to meet various possible use requirements in the subsequent processing process.

In an optional mode of the disclosure, the evaluation policy includes a first evaluation policy, the first evaluation policy is current alarm information or alarm information similar to the current alarm information does not appear after the current code change operation is completed, a sub-effect evaluation index corresponding to the first evaluation policy is a first sub-effect evaluation index, and the first sub-effect evaluation index has a forward influence on a deployment effect of the code module to be evaluated.

In the embodiment of the disclosure, when the current alarm information or the alarm information similar to the current alarm information does not appear after the current code change operation is completed, it indicates that the occurrence of the current alarm information is more likely to be unrelated to the code change operation in the current deployment stage. The first evaluation policy may be configured such that the current alert information or alert information similar to the current alert information does not occur after the current code change operation is completed. The first sub-effect evaluation index corresponding to the first evaluation strategy has a positive effect on the deployment effect of the code module to be evaluated, namely the first sub-effect evaluation index can make the deployment effect tend to be better in deployment effect.

Optionally, when matching is performed on the first evaluation policy, the following manner may be specifically adopted in the embodiment of the present disclosure:

matching a target evaluation strategy in preset evaluation strategies based on the change related information and/or the current alarm information, wherein the target evaluation strategy comprises at least one of the following:

if the current alarm information in the current equipment room appears before the current code change operation and appears for multiple times after the current code change operation is completed, determining a first evaluation strategy as a target evaluation strategy;

if it is determined that first alarm information similar to the current alarm information in the current equipment room appears in a preset time period before the current code changing operation based on the starting and ending time of the current code changing operation, the current equipment room corresponding to the current deployment stage and the current alarm information, determining a first evaluation strategy as a target evaluation strategy;

and if it is determined that second alarm information similar to the current alarm information appears in equipment rooms except the current equipment room based on the current equipment room corresponding to the current deployment stage and the current alarm information, determining the first evaluation strategy as a target evaluation strategy.

In the embodiment of the present disclosure, the first evaluation policy may be matched according to the start-stop time of the current code change operation, the current equipment room corresponding to the current deployment stage, and the current alarm information.

Specifically, when the current alarm information appears before the current code change operation in the current equipment room and appears for multiple times after the current code change operation is completed, the appearance of the current alarm information is considered to be unrelated to the code change operation in the current deployment stage, and at this time, the first evaluation policy may be determined as the target evaluation policy.

Specifically, when first alarm information similar to the current alarm information in the current equipment room appears in a preset time period before the current code change operation, the appearance of the current alarm information is considered to be unrelated to the code change operation in the current deployment stage, and at this time, the first evaluation policy may be determined as the target evaluation policy.

In the embodiment of the present disclosure, since each deployment stage corresponds to different equipment rooms, and the deployment time of different deployment stages is generally different, the first evaluation policy may be matched according to the current equipment room corresponding to the current deployment stage and the current alarm information.

Specifically, when second alarm information similar to the current alarm information has occurred in an equipment room other than the current equipment room, the second alarm information similar to the current alarm information is considered not to occur after completion of the current code change operation in the current deployment stage, and it may be considered that the occurrence of the second alarm information is not related to the current code change operation, and the first evaluation policy may be determined as the target evaluation policy.

As an example, the similarity of the alert information may be determined according to a proportion of similar character strings in the alert information, such as a proportion of similar character strings greater than a certain predetermined proportion.

In an optional manner of the present disclosure, the evaluation policy includes a second evaluation policy, the second evaluation policy is that an influence condition of the current alarm information satisfies a preset influence condition, a sub-effect evaluation index corresponding to the second evaluation policy is a second sub-effect evaluation index, and the second sub-effect evaluation index has a positive influence on a deployment effect of the code module to be evaluated.

In the embodiment of the present disclosure, when the influence of the current alarm information on the operation of each code module in the current deployment stage is small, in order to improve the processing efficiency, it may still be considered that the current alarm information has a positive influence on the deployment effect in the current deployment stage. The preset influence condition can be configured, and when the current alarm information meets the preset influence condition, the influence of the current alarm information on the operation of each code module in the current deployment stage is smaller. And taking the influence condition of the current alarm information meeting the preset influence condition as a second evaluation strategy. The second sub-effect evaluation index corresponding to the second evaluation strategy has a positive effect on the deployment effect of the code module to be evaluated, namely the second sub-effect evaluation index can make the deployment effect tend to be better in deployment effect.

Optionally, when matching is performed for the second evaluation policy, the following manner may be specifically adopted in the embodiment of the present disclosure.

if the number of equipment with current alarm information in the current equipment machine room meets a first preset number condition based on the current equipment machine room corresponding to the current deployment stage and the current alarm information, determining a second evaluation strategy as a target evaluation strategy;

and if the number of the current equipment rooms with the current alarm information is determined to be not satisfied with a second preset number condition or the number of the equipment with the current alarm information in the current equipment room is determined to be not satisfied with a third preset number condition based on the current equipment rooms corresponding to the current deployment stage and the current alarm information, determining a second evaluation strategy as a target evaluation strategy.

In the embodiment of the present disclosure, the second evaluation policy may be matched according to the current equipment room corresponding to the current deployment stage and the current alarm information.

Specifically, a first quantity condition may be preconfigured for the number of devices in which current alarm information occurs in the current device room, and when the number of devices in which current alarm information occurs in the current device room meets the first preset quantity condition, it indicates that the influence of the current alarm information on the operation of each code module in the current deployment stage is small, and at this time, the second evaluation policy may be determined as the target evaluation policy.

As an example, the first preset number condition may be that the proportion of the number of devices in which the current alarm information occurs in the current device room to the total number of devices in the current device room is not greater than 5%.

Specifically, a second preset number condition may be configured for the number of current equipment rooms in which current alarm information occurs, and a third preset number condition may be configured for the number of equipment in the current equipment room in which current alarm information occurs. And if the number of the current equipment rooms with the current alarm information meets a second preset number condition, indicating that the proportion of the current alarm information appearing in the current equipment rooms is higher. If the number of the equipment with the current alarm information in the current equipment room meets a third preset number condition, the proportion of the equipment with the current alarm information in each current equipment room is shown to have a large difference.

When the number of the current equipment rooms in which the current alarm information occurs meets a second preset number condition and the number of the equipment in which the current alarm information occurs in the current equipment room meets a third preset number condition, it is indicated that the proportion of the current alarm information occurring in the current equipment room is higher and the proportion of the equipment in which the current alarm information occurs in each current equipment room is greatly different, at this time, it can be considered that the influence of the current alarm information on the operation of each code module in the current deployment stage is larger, correspondingly, when the number of the current equipment rooms in which the current alarm information occurs does not meet the second preset number condition or the number of the equipment in which the current alarm information occurs in the current equipment room does not meet the third preset number condition, it can be considered that the influence of the current alarm information on the operation of each code module in the current deployment stage is smaller, and at this time, the second evaluation strategy can be determined as a target evaluation strategy.

As an example, the second preset number condition may be that the number of the current equipment rooms in which the current alarm information has occurred accounts for not less than 80% of the total number of the current equipment rooms. The third preset quantity condition may be that the proportions of the quantity of the equipment with the current alarm information appearing in each current equipment room to the total quantity of the equipment in the corresponding equipment room are different.

In an optional manner of the disclosure, the evaluation policy includes a third evaluation policy, where the third evaluation policy is that a similarity between the current alarm information and the alarm information in the first target historical evaluation record satisfies a first preset similar condition, or a similarity between a change content corresponding to the current code change operation and a change content in the first target historical evaluation record satisfies a second preset similar condition, the first target historical evaluation record is a historical evaluation record in which a deployment effect of a corresponding code module satisfies a preset forward effect condition, a sub-effect evaluation index corresponding to the third evaluation policy is a third sub-effect evaluation index, and the third sub-effect evaluation index has a forward influence on the deployment effect of the code module to be evaluated.

In the embodiment of the disclosure, the historical evaluation record generally includes a manual evaluation result of the deployment effect, and the reference to the manual evaluation result in the historical evaluation record can effectively evaluate the deployment effect of the code module to be evaluated. Therefore, a historical knowledge base can be constructed according to the historical evaluation records, and the deployment effect of the code module can be evaluated by accessing the historical database.

In the embodiment of the present disclosure, the result of manual evaluation in the history evaluation record may be a positive influence on the deployment effect or a negative influence on the deployment effect.

For example, the manual evaluation result may include "pass" of the manual annotation, indicating a positive impact on the deployment effect, and the manual evaluation result may also include "no pass" of the manual annotation, indicating a negative impact on the deployment effect.

For another example, the result of the manual evaluation may include a manually labeled alarm reason, and when the manually labeled alarm reason is a low risk reason, such as an equipment reason, a traffic reason, an accidental reason, and the like, the deployed code module may still be considered to have a positive impact on the deployment effect when the alarm reason is the low risk reason because the impact on the operation effect of the deployed code module is small.

In the embodiment of the present disclosure, the history evaluation record having a positive influence on the deployment effect may be used as the first target history evaluation record, and the matching of the third evaluation policy may be performed based on the first target history evaluation record. And when certain similarity exists between the current code changing operation and the first target historical evaluation record, evaluating the deployment effect of the code module to be evaluated by referring to the first target historical evaluation record.

Specifically, a first preset similar condition may be configured for the alarm information, and a second preset similar condition may be configured for the change content. The similarity between the current alarm information and the alarm information in the first target historical evaluation record meets a first preset similar condition, or the similarity between the change content corresponding to the current code change operation and the change content in the first target historical evaluation record meets a second preset similar condition, which indicates that the certain similarity exists between the current code change operation and the first target historical evaluation record, the deployment effect of the code module to be evaluated can be evaluated by referring to the first target historical evaluation record, and the influence of a third sub-effect evaluation index corresponding to a third evaluation strategy on the deployment effect of the code module to be evaluated is set to have a positive influence, namely the third sub-effect evaluation index can enable the deployment effect to tend to be better.

As an example, the alert information and the change content generally satisfy specific data formats, and the alert information and the change content may be divided into character strings according to the specific data formats. The first preset similar condition may be that the proportion of the same character strings in the alarm information is greater than a first preset value, and the second preset similar condition may be that the proportion of the same character strings in the changed content is greater than a second preset value.

As an example, to ensure the effectiveness of the historical evaluation record, the deployment effect of the code module to be evaluated may be evaluated with reference to the historical evaluation record in the adjacent period. For example, the immediate period may be 24 hours prior to the current code change operation.

In an optional manner of the present disclosure, the evaluation policy includes a fourth evaluation policy, where the fourth evaluation policy is that the current alarm information or the device identifier of the current device belongs to a preset white list, a sub-effect evaluation index corresponding to the fourth evaluation policy is a fourth sub-effect evaluation index, and the fourth sub-effect evaluation index has a forward influence on the deployment effect of the code module to be evaluated.

In the embodiment of the present disclosure, the white list may be configured manually, so that the information contained in the white list has a positive influence on the deployment effect.

Specifically, the information included in the white list may include the name, content, device identification, and the like of the alarm information. The fourth evaluation policy may be configured such that the current alarm information or the device identifier of the current device belongs to a preset white list. The fourth sub-effect evaluation index corresponding to the fourth evaluation strategy has a positive effect on the deployment effect of the code module to be evaluated, that is, the fourth sub-effect evaluation index can make the deployment effect tend to be better in deployment effect.

As an example, when at least one of a case where the name of the current alarm information exists in a white list, a case where part of alarm contents included in the current alarm information exists in the white list, a case where the device identification of the current device exists in the white list, and the like occurs, it may be determined that the fourth evaluation policy is matched as the target evaluation policy.

As an example, to guarantee the validity of the white list, a validity period may be set for the white list, for example, the validity period of the white list is set to 24 hours.

In an optional manner of the disclosure, the evaluation policy includes a fifth evaluation policy, where the fifth evaluation policy is that the current alarm information includes designated alarm content, or change content corresponding to a code change operation of the code module to be evaluated includes designated change content, both the designated alarm content and the designated change content are determined based on a second target history evaluation record, the second target history evaluation record is a history evaluation record in which a deployment effect of the corresponding code module satisfies a preset negative effect condition, a sub-effect evaluation index corresponding to the fifth evaluation policy is a fifth sub-effect evaluation index, and the fifth sub-effect evaluation index has a negative influence on the deployment effect of the code module to be evaluated.

In the embodiment of the disclosure, the manual evaluation result in the history evaluation record may be a negative influence on the deployment effect, the history evaluation record having the negative influence on the deployment effect may be used as a second target history evaluation record, analysis is performed based on the second target history evaluation record, specified alarm content or specified change content is determined from the second target history evaluation record, and the specified alarm content or the specified change content can reflect that the deployment effect of the corresponding code module is not good.

Specifically, the alarm information corresponding to the plurality of second target historical evaluation records may be analyzed, and when there are alarm contents appearing many times in the alarm information, it may be considered that the deployment effect of the corresponding code module is not good due to the alarm contents, and the alarm contents are used as the designated alarm contents.

For example, the alert information may include an alert log in which certain page identifiers occur multiple times or certain A/B test identifiers occur multiple times, and these page identifiers or A/B test identifiers may be used as the designated alert content.

In the embodiment of the present disclosure, the change content corresponding to the second target historical evaluation record may be analyzed.

As an example, the static code corresponding to the configuration information in the change content may be analyzed to obtain a call relation tree, obtain a nested relation configured in the call relation tree, bind the change content with the alarm information according to the nested relation, that is, determine the change content that causes the alarm, and use the bound change content as the designated change content.

As another example, a vocabulary in the modified content may be analyzed, the vocabulary in the code is generally used as a variable incoming parameter, a call link analysis may be performed, the alarm information is bound to the changed parameter according to the call link analysis result, and the vocabulary corresponding to the bound parameter is used as the specified modified content.

In the embodiment of the present disclosure, the fifth evaluation policy may be configured that the current alarm information includes specified alarm content, or that the change content corresponding to the code change operation of the code module to be evaluated includes specified change content. The fifth sub-effect evaluation index corresponding to the fifth evaluation strategy has a negative influence on the deployment effect of the code module to be evaluated, that is, the fifth sub-effect evaluation index can make the deployment effect tend to be poor.

In the embodiment of the disclosure, the data accumulated in the historical knowledge base can be copied periodically by manpower, for some alarm information with less influence on the operation effect of the code module, the importance degree of the alarm information in the alarm condition can be reduced by adjusting the alarm condition, and the code module only generating the alarm information is prevented from being brought into the module to be evaluated, so that the evaluation of the deployment effect evaluation for the code modules is also avoided, the workload of the evaluation is reduced, and the processing efficiency is improved.

In the embodiment of the disclosure, when the historical evaluation records are labeled manually, the labeling of the alarm reason can be increased, so that the subsequent enrichment of the evaluation strategy is facilitated, and the accuracy and the effectiveness of the evaluation are improved.

In the embodiment of the disclosure, code change on the line may be initiated on multiple change platforms, and a historical knowledge base obtained by manually marking a certain change platform may be expanded to multiple change platforms for use, so as to improve accuracy and effectiveness of evaluating deployment effects.

As an example, an architecture diagram of a deployment effectiveness evaluation system provided by an embodiment of the present disclosure is shown in fig. 2.

As shown in fig. 2, the experiment deployment is that deployment of a code module is performed within an experiment environment. The light deployment, i.e. the deployment of light applications, is a deployment mode that can be used alternatively in the present system, and this deployment mode may not be used in this example. And (4) completing the modules, namely completing the supplement of each module in the system. And releasing the code corresponding to the system to be deployed. The gray release refers to the release mode which selects the gray release mode for deployment when data is released, and the gray release (named canary release) refers to the release mode which can be smoothly transited between black and white. BP, business process (business process). The single machine room and the multiple machine rooms refer to that in the service process, code modules are firstly deployed in the single machine room and then deployed in the multiple machine rooms. After the grey scale release is completed, the code is deployed in a commercial retrieval system, which is an example of a system to be deployed.

CORE, FATAL log information, wherein the CORE (CORE) log, the FATAL error (FATAL) log, is an alarm log generated in a commercial retrieval system or a system in a gray scale release stage after deployment is completed. A code module generating alarm logs such as a CORE (CORE) log, a FATAL error (FATAL) log and the like is used as a module to be evaluated, and the deployment effect of the module needs to be evaluated.

And (3) interrupting the alarm, namely interrupting the deployment process of the corresponding code module after detecting alarm logs such as a CORE (CORE) log, a FATAL error (FATAL) log and the like, and evaluating the deployment effect in the current deployment stage.

The intelligent diagnosis knowledge base, i.e. the database containing the evaluation strategy in this example, is used for evaluating the deployment effect of the module to be evaluated.

And (4) automatic recovery, namely when the deployment effect of the module to be evaluated is better, the module to be evaluated can be recovered to be in a normal running state, and the module to be evaluated is allowed to be changed in a subsequent deployment stage.

The annotation platform, i.e. the platform for manually annotating the deployment effect, can be used for manual annotation by an administrator (OPERATOR, OP) and an engineer.

The deployment effect evaluation system provided by the embodiment of the disclosure can automatically evaluate deployment effects, reduces dependence on manual evaluation, and is beneficial to improving processing efficiency. As an example, fig. 3 and fig. 4 show a trend chart of a change of a relevant index in an operation process of the deployment effectiveness evaluation system provided by the embodiment of the present disclosure.

As shown in fig. 3, the call amount, i.e., the number of matched target evaluation policies. The exemption rate is the ratio of the code modules to be evaluated, which are evaluated to have better deployment effect, in all the code modules to be evaluated. Exempt from the rate of accuracy, will be assessed as the percentage of the code module that the actual deployment effect still is the preferred in the code module that deployment effect is the preferred. The actual deployment effect can be obtained by a manual marking mode.

In fig. 3, a column in the bar chart indicates the amount of call, the curve located at the top in fig. 3 is an exemption accuracy curve, and the curve located below the exemption accuracy curve is an exemption rate curve.

The

values

0, 20, 40, 60, 80, 100, 120 on the left side of fig. 3 are coordinates of the exemption accuracy and the exemption rate, and the

values

1000, 600, 3000, 4000, 5000, 6000 on the right side of fig. 3 are coordinates of the call amount. The time coordinates of the deployment effectiveness evaluation system during operation, representing the months, are 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 on the lower side in fig. 3.

As can be seen from the dynamic relationship among the call amount, the exemption rate, and the exemption accuracy shown in fig. 3, as the operation time of the deployment effect evaluation system increases, the exemption rate is gradually increased, and the exemption accuracy can be maintained at a higher level.

As shown in fig. 4, the number of alarms, i.e., the number of code modules to be evaluated by human evaluation, is manually processed. And (4) manually processing the alarm proportion, namely the proportion of the number of the code modules to be evaluated, which are manually evaluated, in the total number of the code modules to be evaluated.

In fig. 4, the columns in the bar chart represent the number of manual handling alarms, and the curves in fig. 4 are curves of the manual handling alarm ratio.

The left-hand values 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 in fig. 4 are coordinates of the proportion of the manually processed alarms, and the right-

hand values

10, 20, 30, 40, 50, 60, 70, 80, 90 in fig. 4 are coordinates of the number of manually processed alarms. In the lower part of fig. 4, 13 th week 2021, 14 th week 2021, 15 th week 2021, \8230 \ 8230;, 33 th week 2021, 34 th week 2021, and 35 th week 2021 deploy time coordinates of the effectiveness evaluation system during operation.

As can be seen from the dynamic relationship between the number of human handling alarms and the human handling alarm ratio shown in fig. 4, the human handling alarm ratio is at a higher value in the initial period of operation, then decreases as the operation time of the deployment effectiveness evaluation system increases, and is always stabilized at a lower value.

Based on the same principle as the method shown in fig. 1, fig. 5 shows a schematic structural diagram of a deployment effect evaluation apparatus provided by the embodiment of the disclosure, and as shown in fig. 5, the deployment effect evaluation apparatus 50 may include:

an information obtaining module 510, configured to obtain change-related information corresponding to a current code change operation performed by a code module to be evaluated in a current deployment stage, and/or obtain current alarm information of the code module to be evaluated in the current deployment stage, where the code module to be evaluated is a code module whose current alarm information meets a preset alarm condition;

and the deployment effect evaluation module 520 is used for determining the deployment effect of the code module to be evaluated based on the change related information and/or the current alarm information.

According to the device provided by the embodiment of the disclosure, the deployment effect of the code module to be evaluated is determined by determining the code module to be evaluated of which the current alarm information meets the preset alarm condition, based on the change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage, and/or the current alarm information of the code module to be evaluated in the current deployment stage. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is favorably reduced, and the processing efficiency is improved.

Optionally, the deployment effect evaluation module is specifically configured to:

Optionally, the evaluation policy includes at least two items, the evaluation policy corresponds to a preset matching sequence, and the deployment effect evaluation module is specifically configured to, when a target evaluation policy is matched in the preset evaluation policies based on the change related information and/or the current alarm information:

Optionally, the change-related information includes:

starting and stopping time of current code change operation;

a current equipment room corresponding to the current deployment stage;

the corresponding change content of the current code change operation;

and the current equipment of the current alarm information appears.

Optionally, the evaluation policy includes a first evaluation policy, the first evaluation policy is that the current alarm information or the alarm information similar to the current alarm information does not appear after the current code change operation is completed, a sub-effect evaluation index corresponding to the first evaluation policy is a first sub-effect evaluation index, and the first sub-effect evaluation index has a positive influence on a deployment effect of the code module to be evaluated.

Optionally, the deployment effect evaluation module is specifically configured to, when a target evaluation policy is matched in the preset evaluation policies based on the change related information and/or the current alarm information, at least one of the following:

if the current alarm information in the current equipment room appears before the current code changing operation and appears for multiple times after the current code changing operation is completed, determining a first evaluation strategy as a target evaluation strategy;

if it is determined that first alarm information similar to the current alarm information in the current equipment room appears in a preset time period before the current code change operation based on the start-stop time of the current code change operation, the current equipment room corresponding to the current deployment stage and the current alarm information, determining a first evaluation strategy as a target evaluation strategy;

and if second alarm information similar to the current alarm information is determined to appear in equipment rooms except the current equipment room based on the current equipment room corresponding to the current deployment stage and the current alarm information, determining the first evaluation strategy as a target evaluation strategy.

Optionally, the evaluation policy includes a second evaluation policy, the second evaluation policy is that an influence condition of the current alarm information satisfies a preset influence condition, a sub-effect evaluation index corresponding to the second evaluation policy is a second sub-effect evaluation index, and the second sub-effect evaluation index has a forward influence on a deployment effect of the code module to be evaluated.

if the number of the devices with the current alarm information in the current device room is determined to meet a first preset number condition based on the current device room corresponding to the current deployment stage and the current alarm information, determining a second evaluation strategy as a target evaluation strategy;

Optionally, the evaluation policy includes a third evaluation policy, where the third evaluation policy is that the similarity between the current alarm information and the alarm information in the first target historical evaluation record satisfies a first preset similar condition, or the similarity between the change content corresponding to the current code change operation and the change content in the first target historical evaluation record satisfies a second preset similar condition, the first target historical evaluation record is a historical evaluation record in which the deployment effect of the corresponding code module satisfies a preset forward effect condition, the sub-effect evaluation index corresponding to the third evaluation policy is a third sub-effect evaluation index, and the third sub-effect evaluation index has a forward influence on the deployment effect of the code module to be evaluated.

Optionally, the evaluation policy includes a fourth evaluation policy, the fourth evaluation policy is that the current alarm information or the device identifier of the current device belongs to a preset white list, a sub-effect evaluation index corresponding to the fourth evaluation policy is a fourth sub-effect evaluation index, and the fourth sub-effect evaluation index has a positive influence on the deployment effect of the code module to be evaluated.

Optionally, the evaluation policy includes a fifth evaluation policy, where the fifth evaluation policy is that the current alarm information includes specified alarm content, or that change content corresponding to a code change operation of the code module to be evaluated includes specified change content, both the specified alarm content and the specified change content are determined based on a second target historical evaluation record, the second target historical evaluation record is a historical evaluation record in which a deployment effect of the corresponding code module meets a preset negative effect condition, a sub-effect evaluation index corresponding to the fifth evaluation policy is a fifth sub-effect evaluation index, and the fifth sub-effect evaluation index has a negative effect on the deployment effect of the code module to be evaluated.

It can be understood that each module of the deployment effect evaluation apparatus in the embodiment of the present disclosure has a function of implementing a corresponding step of the deployment effect evaluation method in the embodiment shown in fig. 1. The function can be realized by hardware, and can also be realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above. The modules may be software and/or hardware, and each module may be implemented separately or implemented by integrating a plurality of modules. For the functional description of each module of the deployment effect evaluation apparatus, reference may be specifically made to the corresponding description of the deployment effect evaluation method in the embodiment shown in fig. 1, and details are not repeated here.

In the technical scheme of the disclosure, the processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the common customs of public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

The electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the deployment effectiveness evaluation methods as provided by the embodiments of the present disclosure.

Compared with the prior art, the electronic equipment determines the deployment effect of the code module to be evaluated by determining the code module to be evaluated of which the current alarm information meets the preset alarm condition, based on the change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage, and/or the current alarm information of the code module to be evaluated in the current deployment stage. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is favorably reduced, and the processing efficiency is improved.

The readable storage medium is a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the deployment effectiveness evaluation method provided by the embodiments of the present disclosure.

Compared with the prior art, the readable storage medium determines the deployment effect of the code module to be evaluated by determining the code module to be evaluated of which the current alarm information meets the preset alarm condition, based on the change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage, and/or the current alarm information of the code module to be evaluated in the current deployment stage. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is reduced, and the processing efficiency is improved.

The computer program product comprises a computer program which, when executed by a processor, implements the deployment effectiveness assessment method as provided by the embodiments of the present disclosure.

Compared with the prior art, the computer program product determines the deployment effect of the code module to be evaluated by determining the code module to be evaluated, of which the current alarm information meets the preset alarm condition, based on the change related information corresponding to the current code change operation of the code module to be evaluated in the current deployment stage and/or the current alarm information of the code module to be evaluated in the current deployment stage. Based on the scheme, the deployment effect of the code modules can be automatically and effectively evaluated, the dependence degree on manpower is reduced, and the processing efficiency is improved.

FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the electronic device 600 includes a computing unit 610 that may perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 620 or a computer program loaded from a storage unit 680 into a Random Access Memory (RAM) 630. In the RAM 630, various programs and data required for the operation of the device 600 can also be stored. The computing unit 610, the ROM 620, and the RAM 630 are connected to each other by a bus 640. An input/output (I/O) interface 650 is also connected to bus 640.

Various components in device 600 are connected to I/O interface 650, including: an input unit 660 such as a keyboard, a mouse, etc.; an output unit 670 such as various types of displays, speakers, and the like; a storage unit 680 such as a magnetic disk, optical disk, or the like; and a communication unit 690 such as a network card, modem, wireless communication transceiver, etc. The communication unit 690 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 610 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 610 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 610 performs the deployment effect evaluation method provided in the embodiments of the present disclosure. For example, in some embodiments, performing the deployment effectiveness evaluation methods provided in embodiments of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 680. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 620 and/or the communication unit 690. When the computer program is loaded into RAM 630 and executed by computing unit 610, one or more steps of the deployment effectiveness evaluation methods provided in embodiments of the present disclosure may be performed. Alternatively, in other embodiments, the computing unit 610 may be configured in any other suitable manner (e.g., by way of firmware) to perform the deployment effectiveness evaluation methods provided in embodiments of the present disclosure.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, causes the functions/acts specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server combining a blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A deployment effectiveness assessment method, comprising:

2. The method according to claim 1, wherein the determining, based on the change-related information and/or the current-alert information, a deployment effect of the code module to be evaluated includes:

3. The method according to claim 2, wherein the evaluation policy includes at least two items, the evaluation policy corresponds to a preset matching order, and the matching of the target evaluation policy in the preset evaluation policy based on the change-related information and/or the current alarm information includes:

4. The method of claim 2 or 3, wherein the change-related information comprises:

the start-stop time of the current code change operation;

the current equipment room corresponding to the current deployment stage;

the corresponding change content of the current code change operation;

and the current equipment of the current alarm information appears.

5. The method according to claim 4, wherein the evaluation policy includes a first evaluation policy, the first evaluation policy is that the current alarm information or alarm information similar to the current alarm information does not appear after the current code changing operation is completed, a sub-effect evaluation index corresponding to the first evaluation policy is a first sub-effect evaluation index, and the first sub-effect evaluation index has a positive influence on a deployment effect of the code module to be evaluated.

6. The method according to claim 5, wherein the matching out of the preset evaluation policies a target evaluation policy based on the change related information and/or the current alarm information comprises at least one of:

if it is determined that the current alarm information appears in the current equipment room before the current code changing operation and the current alarm information appears for multiple times after the current code changing operation is completed based on the starting and ending time of the current code changing operation, the current equipment room corresponding to the current deployment stage and the current alarm information, determining the first evaluation strategy as a target evaluation strategy;

if it is determined that first alarm information similar to the current alarm information in the current equipment room appears in a preset time period before the current code changing operation based on the starting and ending time of the current code changing operation, the current equipment room corresponding to the current deployment stage and the current alarm information, determining the first evaluation strategy as a target evaluation strategy;

7. The method according to any one of claims 4 to 6, wherein the evaluation policy includes a second evaluation policy, the second evaluation policy is that an influence condition of the current alarm information satisfies a preset influence condition, a sub-effect evaluation index corresponding to the second evaluation policy is a second sub-effect evaluation index, and the second sub-effect evaluation index has a positive influence on a deployment effect of the code module to be evaluated.

8. The method according to claim 7, wherein the matching out of preset evaluation policies a target evaluation policy based on the change-related information and/or the current alarm information comprises at least one of:

if the number of the devices with the current alarm information in the current device room is determined to meet a first preset number condition based on the current device room corresponding to the current deployment stage and the current alarm information, determining the second evaluation strategy as a target evaluation strategy;

and if the number of the current equipment rooms with the current alarm information is determined to be not satisfied with a second preset number condition or the number of the equipment with the current alarm information in the current equipment room is determined to be not satisfied with a third preset number condition based on the current equipment room corresponding to the current deployment stage and the current alarm information, determining the second evaluation strategy as a target evaluation strategy.

9. The method according to any one of claims 4 to 8, wherein the evaluation policies include a third evaluation policy, the third evaluation policy is that the similarity between the current alarm information and the alarm information in the first target historical evaluation record satisfies a first preset similarity condition, or the similarity between the change content corresponding to the current code change operation and the change content in the first target historical evaluation record satisfies a second preset similarity condition, the first target historical evaluation record is a historical evaluation record in which the deployment effect of the corresponding code module satisfies a preset forward effect condition, the sub-effect evaluation index corresponding to the third evaluation policy is a third sub-effect evaluation index, and the third sub-effect evaluation index has a forward effect on the deployment effect of the code module to be evaluated.

10. The method according to any one of claims 4 to 9, wherein the evaluation policy includes a fourth evaluation policy, the fourth evaluation policy is that the current alarm information or the device identifier of the current device is included in a preset white list, a sub-effect evaluation index corresponding to the fourth evaluation policy is a fourth sub-effect evaluation index, and the fourth sub-effect evaluation index has a positive effect on the deployment effect of the code module to be evaluated.

11. The method according to any one of claims 4 to 10, wherein the evaluation policy includes a fifth evaluation policy, the fifth evaluation policy is that the current alarm information includes specified alarm content, or that change content corresponding to a code change operation of the code module to be evaluated includes specified change content, both the specified alarm content and the specified change content are determined based on a second target historical evaluation record, the second target historical evaluation record is a historical evaluation record in which a deployment effect of the corresponding code module satisfies a preset negative effect condition, a sub-effect evaluation index corresponding to the fifth evaluation policy is a fifth sub-effect evaluation index, and the fifth sub-effect evaluation index has a negative effect on the deployment effect of the code module to be evaluated.

12. A deployment effectiveness evaluation apparatus comprising:

the system comprises an information acquisition module, a code module and a code module, wherein the information acquisition module is used for acquiring change related information corresponding to current code change operation performed by the code module to be evaluated in a current deployment stage, and/or current alarm information of the code module to be evaluated in the current deployment stage, and the code module to be evaluated is a code module of which the current alarm information meets preset alarm conditions;

13. The apparatus according to claim 12, wherein the deployment effectiveness evaluation module is specifically configured to:

matching a target evaluation strategy in preset evaluation strategies based on the change related information and/or the current alarm information, wherein the evaluation strategies are preconfigured with corresponding sub-effect evaluation indexes;

14. The apparatus according to claim 13, wherein the evaluation policy includes at least two items, the evaluation policy corresponds to a preset matching order, and the deployment effectiveness evaluation module is specifically configured to, when a target evaluation policy is matched in the preset evaluation policies based on the change-related information and/or the current alarm information:

15. The apparatus of claim 13 or 14, wherein the change-related information comprises:

the start-stop time of the current code change operation;

the current equipment room corresponding to the current deployment stage;

the corresponding change content of the current code change operation;

and the current equipment of the current alarm information appears.

16. The apparatus of claim 15, wherein the evaluation policy includes a first evaluation policy, the first evaluation policy is that the current alarm information or alarm information similar to the current alarm information does not appear after the current code change operation is completed, and a sub-effect evaluation index corresponding to the first evaluation policy is a first sub-effect evaluation index, and the first sub-effect evaluation index has a positive effect on a deployment effect of the code module to be evaluated.

17. The apparatus according to claim 16, wherein the deployment effectiveness evaluation module, when matching a target evaluation policy among preset evaluation policies based on the change-related information and/or the current alarm information, is specifically configured to at least one of:

18. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

19. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-11.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-11.