WO2020140684A1 - 评估车损识别模型的方法及装置 - Google Patents
评估车损识别模型的方法及装置 Download PDFInfo
- Publication number
- WO2020140684A1 WO2020140684A1 PCT/CN2019/123344 CN2019123344W WO2020140684A1 WO 2020140684 A1 WO2020140684 A1 WO 2020140684A1 CN 2019123344 W CN2019123344 W CN 2019123344W WO 2020140684 A1 WO2020140684 A1 WO 2020140684A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- damage
- objects
- test
- test sample
- identification model
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
Definitions
- One or more embodiments of this specification relate to the field of machine learning, and in particular, to a method and device for evaluating a vehicle damage recognition model.
- the insurance company needs to send professional damage survey personnel to the accident site to conduct on-site damage assessment, give the vehicle's maintenance plan and compensation amount, and take photos of the scene, and the photos of the fixed losses are kept for background verification Personnel damage and price. Due to the need to manually investigate and assess damage, insurance companies need to invest a lot of labor costs, and professional knowledge training costs. From the experience of ordinary users, the claims process is due to waiting for human surveyors to take pictures on the spot, the loss adjuster fixes the damage at the repair site, and the loss loss personnel check the damage in the background. The claim period is as long as 1-3 days, and the user has a long wait time , The experience is poor.
- Intelligent image loss determination is based on the computer vision image recognition technology in the field of artificial intelligence. Based on the on-site loss pictures taken by ordinary users, it automatically recognizes the lost parts and the degree of loss reflected in the pictures, and automatically gives a maintenance plan. Finally, by calling the price library corresponding to the insurance company's maintenance program, the compensation amount of the case is obtained. This solution eliminates the need to manually check and assess damage and loss, greatly reducing the cost of insurance companies and improving the average user's experience in auto insurance claims.
- One or more embodiments of this specification describe a method and device for evaluating a vehicle damage identification model, by distinguishing between markedly damaged objects and non-significantly damaged objects in the test sample, to avoid evaluation interference caused by the ambiguity of non-significantly damaged objects .
- a method for evaluating a vehicle damage identification model including:
- the test result of the first test sample by the vehicle damage identification model is determined according to the relationship between the set of predicted damage objects and the set of significant damage objects and the set of non-significant damage objects.
- the plurality of sets of labeling data include at least first labeling data and second labeling data, wherein the first labeling data is generated by labeling personnel with a first labeling ability level, and the second labeling data is composed of The tagging personnel of the second tagging capability level generate tags, and the second tagging capability level is higher than the first tagging capability level.
- each set of labeling data in the plurality of sets of labeling data is generated by the labeling personnel and the verification personnel performing the verification.
- test results include results that are incorrectly predicted or correctly predicted.
- test result of the vehicle damage identification model on the first test sample is determined in the following manner:
- test result of the vehicle damage identification model on the first test sample is a prediction error.
- test result of the vehicle damage identification model on the first test sample is determined in the following manner:
- the vehicle damage identification model is determined.
- the test result of the first test sample is a prediction error.
- the test result includes a single sample test score.
- the single-sample test score is determined in the following manner:
- a single-sample positive score of the vehicle damage identification model for the first test sample is determined.
- the one-sample test score is determined by:
- a single-sample negative score of the vehicle damage identification model for the first test sample is determined.
- a method for evaluating a vehicle damage identification model including:
- test sample set which includes multiple test samples
- the multiple test samples correspond to multiple car damage pictures
- each car damage picture corresponds to multiple sets of annotation data
- the multiple sets of annotation data are based on at least multiple annotation personnel pairs
- the damage object in the car damage picture is generated by marking;
- the method of the first aspect is executed to determine each test result of the vehicle damage identification model for each test sample:
- the evaluation result of the vehicle damage identification model for the test sample set is determined.
- each test result includes a result of correct prediction or prediction error; correspondingly, determining the evaluation result may include determining the proportion of each test result that is predicted to be correct, as the evaluation result.
- each test result includes a single sample test score; correspondingly, determining the evaluation result of the vehicle damage identification model for the test sample set includes, according to the single sample test score included in each test result , Determine the total sample score for the test sample set, and use it as the evaluation result.
- an apparatus for evaluating a vehicle damage identification model including:
- a sample acquisition unit configured to acquire a first test sample, the first test sample corresponding to a first car damage picture and multiple sets of annotation data, the multiple sets of annotation data are based at least on the first car damage picture
- the damaged objects are marked and generated;
- An annotation set determination unit configured to determine an intersection and union of the plurality of sets of annotation data, determine a significant damage object set in the first test sample according to the intersection; and according to the intersection and the union To determine the set of non-significantly damaged objects in the first test sample;
- a prediction set determining unit configured to input the first vehicle damage picture into a pre-trained vehicle damage recognition model to obtain a predicted damage object set composed of the damage objects predicted by the vehicle damage recognition model for the first test sample;
- the test result determination unit is configured to determine the test result of the first test sample by the vehicle damage identification model according to the relationship between the set of predicted damage objects and the set of significant damage objects and the set of non-significant damage objects.
- an apparatus for evaluating a vehicle damage identification model including:
- the sample set acquisition unit is configured to acquire a test sample set, which includes multiple test samples, the multiple test samples correspond to multiple car damage pictures, and each car damage picture corresponds to multiple sets of labeled data, the multiple sets of labeled data It is generated based on at least a number of annotators marking the damaged objects in the car damage picture;
- the test result obtaining unit is configured to determine the test results of the vehicle damage identification model for each test sample using the device of the third aspect for each test sample:
- the evaluation result determination unit is configured to determine the evaluation result of the vehicle damage identification model for the test sample set according to the respective test results.
- a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to perform the methods of the first and second aspects.
- a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the first aspect and the first Two ways.
- multiple sets of labeling data generated by multiple labeling personnel labeling the same test picture are used to determine the significant damage object and the non-significant damage object in the test picture.
- comparing the predicted damage objects and labeled data output by the model in order to evaluate the vehicle damage identification model distinguish between significant damage objects and non-significant damage objects, using different comparison methods, based on the predicted damage objects and the significant damage objects respectively
- the relationship of non-significant damage objects determines the test results of the vehicle damage identification model on the test samples. Due to the distinction between significant damage objects and non-significant damage objects, when evaluating the vehicle damage identification model, these two kinds of marked objects are given different attention and different measurement standards, so as to avoid the ambiguity and non-significant damage objects. The controversy and noise in the evaluation caused by the uncertainty will find a better optimization goal for the model optimization.
- the damage identification of the car body has a certain degree of professionalism. Some reflective stains on the car body are easily confused with the damage, which can easily cause mislabeling or missed labeling by the labeling personnel.
- the car lamp is slightly damaged. Due to the complex structure of the car lamp, the glass material is easy to reflect and other factors It requires a person with strong visual ability and experience to make a more accurate label.
- This method can effectively distinguish between clear errors and ambiguous situations, help test the model's ability to optimize clear errors, and can increase the weight of clear errors in the model to reduce the weight of ambiguous situations, so that the model can better distinguish clear errors Sex.
- This method is also suitable for the evaluation and optimization of other scenes that require professional knowledge and high visual ability, such as the labeling of medical diseases.
- FIG. 1 shows an example of a car damage picture according to an embodiment
- FIG. 2 shows a flowchart of a method for evaluating a vehicle damage identification model according to an embodiment
- Figure 3a shows the first set of annotation data in an example
- Figure 3b shows the second set of annotation data in an example
- Figure 3c shows the third set of annotation data in an example
- Figure 4 shows the relationship between the set of significant damage objects, the set of non-significant damage objects, and the set of predicted damage objects
- FIG. 5 shows a method of evaluating a vehicle damage identification model according to an embodiment
- FIG. 6 shows a schematic block diagram of an evaluation device according to an embodiment
- FIG. 7 shows a schematic block diagram of an evaluation device according to an embodiment.
- the overall concept of the embodiment scheme stems from the applicant's analysis and research on human visual ability.
- FIG. 1 shows an example of a car damage picture according to an embodiment.
- people with normal ordinary visual abilities can observe that there are two scratches on the bumper and fender, so the damage is significant.
- a little opaque scratch damage on the lower left side of the left front headlight of the vehicle, and a little scratch damage on the steel ring are not so significant. Only people with strong visual abilities can observe the corners.
- the annotators who annotate car damage pictures are professionals with long-term visual training. Some of these people will have a strong visual observation ability, can observe and mark some non-significant damage, especially after special training, you can professionally mark lights or small parts.
- the labels with significant damage are easier, and there is basically no omission in labeling, but the labels with non-significant damage will vary from person to person and have some deviations.
- the car damage recognition model is trained. The model will cover the significant damage very well, and it will also have a certain super visual ability to observe the perception ability, so the recognition ability of the model is better than the significant damage Large intersections, partly with non-significant damage.
- the applicant proposes a scheme to distinguish between markedly damaged objects and non-significantly damaged objects in the car damage pictures.
- different judgment standards are made for the significantly damaged objects and non-significantly damaged objects, so as to evaluate the damage recognition model more effectively and avoid unnecessary disputes.
- FIG. 2 shows a flowchart of a method for evaluating a vehicle damage identification model according to an embodiment.
- the method can be performed by any device, device, computing platform, or computing cluster with computing and processing capabilities.
- the method includes at least the following steps: Step 21: Obtain a first test sample, the first test sample corresponds to a first car damage image and multiple sets of annotation data, the multiple sets of annotation data are based on at least multiple annotations The person marks the damaged objects in the first car damage picture to generate; Step 22, determines the intersection and union of the multiple sets of labeled data, and determines the significant damaged objects in the first test sample according to the intersection Set; and according to the difference between the intersection and the union, determine the set of non-significant damage objects in the first test sample; Step 23, input the first car damage picture into a pre-trained car damage recognition A model to obtain a predicted damage object set composed of the damage objects predicted by the vehicle damage identification model for the first test sample; step 24, according to the predicted damage object set and the significant damage object set and the non-significant
- a single test sample is obtained.
- this test sample is referred to as the first test sample.
- the first test sample corresponds to a single car damage picture, hereinafter referred to as the first car damage picture, and multiple sets of annotation data for the car damage picture, wherein the multiple sets of annotation data are based on at least the first car damage
- the damaged objects in the picture are generated by annotation.
- an original car damage picture it can be distributed to multiple annotators to mark the vehicle damage objects respectively, thereby generating multiple sets of annotated data.
- the multiple sets of annotation data together with the car damage pictures constitute a test sample.
- the car damage pictures are distributed to the people with different labeling abilities.
- the labeling ability of the labeling personnel will be graded according to the strength of the visual observation ability of the labeling personnel, and the factors such as the length of training time. Generally, a higher level of annotation ability means a stronger visual observation ability.
- the car damage pictures can be distributed to the labeling personnel with different labeling ability levels, thereby generating multiple sets of labeling data.
- the car damage image can be distributed to the labelers with the first labeling ability level for labeling, thereby generating the first labeling data; the same car damage image can be distributed to the labeling personnel with the second labeling ability level Annotation is performed to generate second annotation data, where the second annotation capability level is higher than the first annotation capability level.
- the multiple sets of labeling data include at least the first labeling data and the second labeling data generated by labelers of different labeling ability levels (first and second ability levels), respectively.
- the above-mentioned first annotation ability level corresponds to ordinary visual ability
- the second annotation ability level corresponds to super visual ability
- the multiple sets of annotation data include data generated by an annotation person with ordinary visual abilities tagging a car damage picture, and data generated by an annotation person with super visual abilities tagging the picture.
- the labeling ability levels can also be divided more carefully, for example, into three levels or even four levels, and the above-mentioned multiple sets of labeling data come from labelers of different labeling ability levels.
- the labeling staff For each set of labeling data, the labeling staff generates the labeling data, and then the verification staff performs verification and confirmation. In this way, to avoid the impact of individual labeling personnel's incorrect labeling on the entire test.
- the car damage picture shown in FIG. 1 is used as the original image, and it is distributed to three tagging personnel to tag the damaged object to obtain 3 sets of tagging data.
- Figures 3a-3c respectively show these three sets of annotation data. It can be seen that the three sets of labeling data are not the same due to the differences in the visual capabilities of different labelers.
- Figure 3a contains two marked damage objects, denoted as A1 and A2 respectively;
- Figure 3b contains marked damage objects A1, A2 and newly added A3;
- Figure 3c contains five marked damage objects , A1, A2, A4, A5 and A6, respectively.
- step 22 the intersection and union of multiple sets of labeled data are determined, and the set of significant damage objects in the first test sample is determined according to the intersection; according to the difference between the intersection and the union, the non-information in the first test sample is determined. Significant damage to the object collection.
- step 22 based on the similarities and differences between the multiple sets of labeled data, it is determined which damaged objects are significant damaged objects and which are non-significant damaged objects.
- the injury should be identified as a significant injury. This corresponds to the intersection of multiple sets of labeled data. Therefore, the damaged object in the intersection of the above-mentioned multiple sets of labeled data is determined as the significant damaged object.
- the objects other than the significantly damaged objects are determined as non-significantly damaged objects, that is, the difference between the union of multiple sets of labeled data and the above intersection is determined as non-significantly damaged objects.
- the damaged object set included in FIG. 3a is ⁇ A1, A2 ⁇
- the damaged object set included in FIG. 3b is ⁇ A1, A2, A3 ⁇
- the damaged object set included in FIG. 3c is ⁇ A1, A2, A4, A5, A6 ⁇ .
- the intersection is ⁇ A1, A2 ⁇
- the union is ⁇ A1, A2, A3, A4, A5, A6 ⁇ . Therefore, in one embodiment, it is determined according to the intersection that for the car damage picture of FIG. 1, the set of significant damage objects is ⁇ A1, A2 ⁇ , and the set of non-significant damage objects is ⁇ A3, A4, A5, A6 ⁇ .
- step 23 the above-mentioned vehicle damage image is input into a pre-trained vehicle damage identification model to obtain a predicted damage object set composed of the damage object predicted by the vehicle damage identification model for the test sample .
- the above vehicle damage recognition model can be implemented based on various machine learning algorithms and using various neural network structures, which is not limited herein.
- the vehicle damage identification model is trained based on the training sample set.
- the training sample set contains a large number of training samples, and each training sample corresponds to a car damage picture and the labeled data for the car damage picture. Using the labeled data as the sample label, you can train the vehicle damage recognition model.
- the picture to be recognized is input into the vehicle damage recognition model, and the model will output the recognition result for the picture, also known as the prediction result, which contains each damage object that is recognized or predicted.
- the model For the first car damage picture as the test sample in the previous step, input it into the already trained car damage recognition model, and the model will output each damage object predicted for the first car damage picture.
- the set of predicted damage objects is called a predicted damage object set.
- step 24 according to the relationship between the predicted damage object set obtained in step 23 and the significant damage object set and the non-significant damage object set obtained in step 22, the test result of the vehicle damage identification model on the first test sample is determined.
- FIG. 4 shows the relationship between the set of significant damage objects, the set of non-significant damage objects, and the set of predicted damage objects.
- a circle 101 represents a set of significant damage objects, corresponding to the intersection of multiple sets of labeled data.
- the circle 102 corresponds to the union of the aforementioned multiple sets of labeled data, and represents all the damaged objects marked. Therefore, the circle 102 completely contains the circle 101, and the circle 101 completely falls into the circle 102.
- the part between the circle 102 and the circle 101 that is, the difference between the union and intersection of the multiple sets of labeled data, represents the non-significant damage object. This part of the damage object corresponds to the ambiguous damage, or only those with super visual ability The damage can be identified.
- the predicted damage object set 103' should be located between the circle 101 and the circle 102, as shown by the dotted line in FIG. 4. That is, the predicted damage object set 103' completely includes the significant damage object set 101, but does not exceed the set 102 of all damage objects, and includes some non-significant damage objects. At this time, it can be considered that the result of the predicted damage object set 103' is correct, and the test result of the vehicle damage identification model for the current test sample is good.
- the test result of the vehicle damage identification model for a single sample is determined, that is, whether the prediction of the vehicle damage identification model for the single sample is correct.
- the test results include the conclusion that the prediction is wrong or the prediction is correct.
- the set of predicted damage objects should completely contain the set of significant damage objects, which means that the vehicle damage identification model is required to predict all significant damage objects. Therefore, in one embodiment, an error type that defines prediction errors is called a first type of error.
- This first type of error corresponds to the situation where the vehicle damage identification model fails to identify or predict a significant damage object somewhere. Since the significant damage object is the damage object that will be observed by people with ordinary visual abilities, in the case that the recognition result of the vehicle damage recognition model shows the above first type of error, the vehicle damage recognition model can be tested against the current first test sample The result is determined to be a prediction error.
- step 24 it can be determined in step 24 whether the set of significant damage objects is a subset of the set of predicted damage objects, that is, whether the set of predicted damage objects completely contains the set of significant damage objects; if the above determination is negative, Then it is determined that the test result of the vehicle damage identification model is a prediction error.
- the set of significant damage objects is ⁇ A1, A2 ⁇ . If the predicted damage object set output by the vehicle damage identification model fails to fully contain ⁇ A1, A2 ⁇ , it is considered that the test result for the test sample is a prediction error.
- a second type of error corresponds to the situation where the damage identified by the vehicle damage identification model exceeds the range of all the damage objects marked. Since the range of all damaged objects (corresponding to the circle 102 in FIG. 4) includes significant damaged objects and non-significant damaged objects, or includes all possible damaged objects, the second result of the recognition of the vehicle damage recognition model appears above In the case of a class error, the test result of the vehicle damage identification model for the current first test sample may be determined as a prediction error.
- step 24 for any damage object in the predicted damage object set, hereinafter referred to as the first damage object, if the first damage object does not belong to the significant damage object set and the non-significant damage object set Any one of them determines that the test result of the vehicle damage identification model on the current test sample is a prediction error.
- the set of significant damage objects is ⁇ A1, A2 ⁇
- the set of non-significant damage objects is ⁇ A3, A4, A5, A6 ⁇
- the union is ⁇ A1, A2 , A3, A4, A5, A6 ⁇ . If the predicted damage object set output by the vehicle damage identification model contains a certain damage B1, it does not belong to any one of the significant damage object set and the non-significant damage object set, or does not belong to the union ⁇ A1, A2, A3, A4, A5, A6 ⁇ , it is considered that the vehicle damage identification model recalled the wrong object, and the test result for the test sample is a prediction error.
- the test result with the correct prediction can be determined correspondingly.
- the test result of the vehicle damage identification model for the current test sample is determined to be correct.
- the test result of the vehicle damage identification model for the current test sample is divided into prediction errors or prediction predictions.
- a test score is used to characterize the test result of the vehicle damage identification model for a single test sample.
- the test score may include a positive score of a single sample.
- the positive score is used to indicate the predicted correct rate, correct number, correctness, etc. for the single sample; the test score may also include a negative score of a single sample, a negative
- the score indicates, for example, the prediction error rate for the single sample, the number of errors, and so on.
- the number of damage objects included in the set of predicted damage objects that belong to the set of significant damage objects can be determined, which is called the first number, and is denoted as N1; it is also determined that the number of damage objects included in the set of predicted damage objects belongs to The number of damaged objects in the non-significantly damaged object set is called the second number and is denoted as N2. Then, based on at least the first number N1 and the second number N2, a single-sample positive score of the vehicle damage identification model for the current test sample is determined.
- the sum of the first number N1 and the second number N2 is determined as the number N of correctly predicted damage objects, and the number N is divided by the total number M of elements in the predicted damage object set, and N/M is taken as the above
- the forward score is used to describe the proportion of the damaged objects that are predicted to be correct.
- the ratio R1 of the first number N1 to the number S1 of elements in the significant damage set is determined, and the ratio R2 of the second number N2 to the number S2 of elements in the non-significant damage set is determined.
- R1 and R2 On average, the single sample correct rate R is obtained as the above positive score, namely:
- a negative score of the vehicle damage identification model for a single test sample is determined.
- the negative score NS can be determined in various ways.
- a certain score is assigned to the first type of error and the second type of error, respectively, and the negative score NS is determined as the sum of the above scores, namely:
- the number of damaged objects whose prediction is wrong is also determined.
- the number of damaged objects included in the set of significant damage objects but not included in the set of predicted damage objects is called a third number, which is denoted as N3, and the number of damage objects in the set of predicted damage objects is determined.
- the number of included damaged objects that are not in the set of significant damage objects nor in the set of non-significant damage objects is called the fourth number and is denoted as N4.
- the above third number and fourth number correspond to the above-mentioned first type error and second type error, respectively, and can be considered as the number of prediction errors. Therefore, the negative score NS of the vehicle damage identification model for the current test sample can be determined according to the third number and the fourth number.
- the negative score can be determined as:
- w3 and w4 are weighting factors.
- the predicted damage object set P is ⁇ A1, A3, A4, A5, B1, B2 ⁇ .
- the damage object B1, B2 is included in the predicted damage object set.
- the above positive score and negative score can also be synthesized to obtain a single sample total score for the current test sample.
- the single-sample test result of the vehicle damage identification model against the current test sample is evaluated in various ways.
- FIG. 5 illustrates a method for evaluating a vehicle damage identification model according to an embodiment.
- the method is used to evaluate the overall recognition effect of the vehicle damage identification model on a plurality of test samples in a test sample set.
- a test sample set is obtained, which includes multiple test samples.
- Each test sample corresponds to a car damage picture, and multiple sets of annotation data, the multiple sets of annotation data are generated based on at least multiple annotators marking the damaged objects in the car damage picture.
- multiple sets of labeling data and multiple labeling personnel please refer to the description of the foregoing step 21, which will not be repeated here.
- step 52 for each test sample, the method shown in FIG. 2 is executed, so as to determine each test result of the vehicle damage identification model for each test sample.
- the method shown in FIG. 2 is executed, so as to determine each test result of the vehicle damage identification model for each test sample.
- the test result for the test sample can be obtained.
- step 53 the evaluation result of the vehicle damage identification model for the test sample set is determined according to the above test results.
- each test result may include a result of correct prediction or incorrect prediction.
- determining the evaluation result of the vehicle damage identification model for the test sample set may include determining the ratio of each test result that is predicted correctly, and using this ratio as the evaluation result.
- a certain threshold for example 80%, it can be considered that the recognition effect of the vehicle damage recognition model meets the requirements and the test passes.
- each test result may include a single sample test score.
- determining the evaluation result of the vehicle damage identification model for the test sample set may include determining the total sample score for the test sample set based on the single-sample test score included in each test result, and using the score as the evaluation result.
- a corresponding score threshold can be set.
- the recognition effect of the vehicle damage recognition model is considered to meet the requirements.
- the single-sample test score is a negative score
- the total sample score is the average of each single-sample score. In such a case, if the total sample score is less than the preset score threshold, the test is considered passed.
- the significant and non-significant damage objects in the test picture are determined.
- the relationship of non-significant damage objects determines the test results of the vehicle damage identification model on the test samples. Due to the distinction between significant damage objects and non-significant damage objects, when evaluating the vehicle damage identification model, different attention and different measurement standards are given to the two labeled objects, thereby avoiding the ambiguity and non-significant damage objects. Controversy and noise caused by uncertainty.
- a device for evaluating a vehicle damage identification model is also provided.
- Fig. 6 shows a schematic block diagram of an evaluation device according to an embodiment.
- the evaluation device can be deployed in any device, device, computing platform, or computing cluster with computing and processing capabilities, and is used to evaluate the recognition results of a single test sample by the vehicle damage identification model.
- the evaluation device 600 includes:
- the sample obtaining unit 61 is configured to obtain a first test sample, the first test sample corresponding to a first car damage picture and multiple sets of annotation data, the multiple sets of annotation data are based at least on the first car damage picture The damaged objects in the label are generated;
- An annotation set determination unit 62 is configured to determine the intersection and union of the multiple sets of annotation data, determine the set of significant damage objects in the first test sample according to the intersection; and according to the intersection and the union The difference between them, determine the set of non-significantly damaged objects in the first test sample;
- the prediction set determining unit 63 is configured to input the first vehicle damage picture into a pre-trained vehicle damage recognition model to obtain a predicted damage object set composed of the damage objects predicted by the vehicle damage recognition model for the first test sample;
- the test result determination unit 64 is configured to determine the test result of the first test sample by the vehicle damage identification model based on the relationship between the set of predicted damage objects and the set of significant damage objects and the set of non-significant damage objects .
- the plurality of sets of labeling data include at least first labeling data and second labeling data, wherein the first labeling data is generated by labeling personnel with a first labeling ability level, and the second labeling data is composed of The tagging personnel of the second tagging capability level generate tags, and the second tagging capability level is higher than the first tagging capability level.
- each set of labeling data in the plurality of sets of labeling data is generated by the labeling personnel and the verification personnel performing the verification.
- test results include results that are incorrectly predicted or correctly predicted.
- test result determination unit 64 is configured to:
- test result of the vehicle damage identification model on the first test sample is a prediction error.
- test result determination unit 64 is configured to:
- the vehicle damage identification model is determined.
- the test result of the first test sample is a prediction error.
- the test result includes a single sample test score.
- test result determination unit 64 is configured to:
- a single-sample positive score of the vehicle damage identification model for the first test sample is determined.
- test result determination unit 64 is configured to:
- a single-sample negative score of the vehicle damage identification model for the first test sample is determined.
- FIG. 7 shows a schematic block diagram of an evaluation device according to an embodiment.
- the evaluation device 700 is used to evaluate the recognition result of the test sample set composed of multiple test samples by the vehicle damage identification model. As shown in FIG. 7, the evaluation device 700 includes:
- the sample set acquisition unit 71 is configured to acquire a test sample set, which includes multiple test samples, the multiple test samples correspond to multiple car damage pictures, and each car damage picture corresponds to multiple sets of annotation data, the multiple sets of annotations
- the data is generated based on at least a number of annotators marking the damaged objects in the car damage picture
- the test result acquisition unit 72 is configured to use the device 600 of FIG. 6 for each test sample to determine each test result of the vehicle damage identification model for each test sample:
- the evaluation result determination unit 73 is configured to determine the evaluation result of the vehicle damage identification model for the test sample set according to the respective test results.
- each of the test results includes a result of correct prediction or prediction error; correspondingly, the evaluation result determination unit 73 is configured to determine the ratio of the correct prediction of each test result as the evaluation result.
- each of the test results includes a single-sample test score; correspondingly, the evaluation result determination unit 73 is configured to determine, based on the single-sample test score included in the respective test results, The total sample score of the test sample set is used as the evaluation result.
- a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to perform the method described in conjunction with FIGS. 2 and 5.
- a computing device including a memory and a processor, where executable code is stored in the memory, and when the processor executes the executable code, the implementation is combined with FIG. 2 and FIG. 5 The method.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Marketing (AREA)
- Artificial Intelligence (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Technology Law (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
一种评估车损识别模型的方法和装置,方法包括,首先获取第一测试样本,该测试样本对应第一车损图片以及多组标注数据,该多组标注数据基于多个标注人员对第一车损图片的标注而产生(21);然后确定多组标注数据的交集和并集,根据交集确定测试样本中的显著损伤对象集合;并根据交集与并集之间的差异,确定测试样本中的非显著损伤对象集合(22)。此外,还将第一车损图片输入车损识别模型,得到车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损失对象集合(23)。于是,可以根据预测损伤对象集合与显著损伤对象集合和非显著损伤对象集合的关系,确定车损识别模型对第一测试样本的测试结果(24)。
Description
本说明书一个或多个实施例涉及机器学习领域,尤其涉及评估车损识别模型的方法及装置。
在传统车险理赔过程中,保险公司需要派出专业的查勘定损人员到事故现场进行现场查勘定损,给出车辆的维修方案和赔偿金额,并拍摄现场照片,定损照片留档以供后台核查人员核损核价。由于需要人工查勘定损,保险公司需要投入大量的人力成本,和专业知识的培训成本。从普通用户的体验来说,理赔流程由于等待人工查勘员现场拍照、定损员在维修地点定损、核损人员在后台核损,理赔周期长达1-3天,用户的等待时间较长,体验较差。
针对传统车险理赔中人工成本巨大的行业痛点,提出了一些智能图像定损方案。智能图像定损,是通过人工智能领域计算机视觉图像识别技术,根据普通用户拍摄的现场损失图片,自动识别图片中反映的损失部件及其损失程度,并自动给出维修方案。最后通过调用保险公司维修方案对应的价格库,获得案件的赔偿金额。这一解决方案无需人工查勘定损核损,大大减少了保险公司的成本,提升了普通用户的车险理赔体验。
为了实现智能图像定损,提出了多种利用机器学习算法的车损识别模型。然而,如何对这些车损识别模型的识别效果进行评估,成为有待解决的问题。
因此,希望能有改进的方案,更加有效地对车损识别模型进行评估。
发明内容
本说明书一个或多个实施例描述了一种评估车损识别模型的方法和装置,通过区分测试样本中的显著损伤对象和非显著损伤对象,避免非显著损伤对象的模糊性带来的评估干扰。
根据第一方面,提供了一种评估车损识别模型的方法,包括:
获取第一测试样本,该第一测试样本对应第一车损图片以及多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损伤对象进行标注而产 生;
确定所述多组标注数据的交集和并集,根据所述交集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;
将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;
根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。
在一个实施例中,所述多组标注数据至少包括,第一标注数据和第二标注数据,其中第一标注数据由具有第一标注能力等级的标注人员标注产生,第二标注数据由具有第二标注能力等级的标注人员标注产生,所述第二标注能力等级高于所述第一标注能力等级。
根据一种实施方案,多组标注数据中每组标注数据通过标注人员进行标注,以及核查人员进行核查而产生。
在一种实施方式中,测试结果包括预测错误或预测正确的结果。
进一步的,在一个实施例中,通过以下方式确定车损识别模型对第一测试样本的测试结果:
判断所述显著损伤对象集合是否为所述预测损伤对象集合的子集;
如果不是其子集,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
在另一实施例中,通过以下方式确定车损识别模型对第一测试样本的测试结果:
对于预测损伤对象集合中任意的第一损伤对象,如果该第一损伤对象不属于所述显著损伤对象集合和所述非显著损伤对象集合中的任一个,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
在另一种实施方式中,测试结果包括单样本测试分数。
进一步的,在一个实施例中,通过以下方式确定单样本测试分数:
确定所述预测损伤对象集合中所包含的、属于所述显著损伤对象集合的损伤对象 的第一数目;
确定所述预测损伤对象集合中所包含的、属于所述非显著损伤对象集合的损伤对象的第二数目;
至少根据所述第一数目和第二数目,确定所述车损识别模型针对所述第一测试样本的单样本正向分数。
在另一实施例中,通过以下方式确定单样本测试分数:
确定包含在所述显著损伤对象集合中、但不包含在所述预测损伤对象集合中的损伤对象的第三数目;
确定所述预测损伤对象集合中所包含的、不属于所述显著损伤对象集合也不属于所述非显著损伤对象集合的损伤对象的第四数目;
根据所述第三数目和第四数目,确定所述车损识别模型针对所述第一测试样本的单样本负向分数。
根据第二方面,提供一种评估车损识别模型的方法,包括:
获取测试样本集,其中包括多个测试样本,所述多个测试样本对应多个车损图片,每个车损图片对应具有多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生;
对于各个测试样本,执行第一方面的方法,从而确定出所述车损识别模型针对各个测试样本的各个测试结果:
根据所述各个测试结果,确定所述车损识别模型针对所述测试样本集的评估结果。
在一个实施例中,各个测试结果包括预测正确或预测错误的结果;相应的,确定评估结果可以包括,确定各个测试结果中预测正确的比例,作为所述评估结果。
在另一实施例中,各个测试结果包括单样本的测试分数;相应的,确定所述车损识别模型针对所述测试样本集的评估结果包括,根据各个测试结果中包括的单样本的测试分数,确定针对所述测试样本集的总样本分数,将其作为所述评估结果。
根据第三方面,提供一种评估车损识别模型的装置,包括:
样本获取单元,配置为获取第一测试样本,该第一测试样本对应第一车损图片以及多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损 伤对象进行标注而产生;
标注集合确定单元,配置为确定所述多组标注数据的交集和并集,根据所述交集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;
预测集合确定单元,配置为将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;
测试结果确定单元,配置为根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。
根据第四方面,提供一种评估车损识别模型的装置,包括:
样本集获取单元,配置为获取测试样本集,其中包括多个测试样本,所述多个测试样本对应多个车损图片,每个车损图片对应具有多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生;
测试结果获取单元,配置为对于各个测试样本,利用第三方面的装置,确定出所述车损识别模型针对各个测试样本的各个测试结果:
评估结果确定单元,配置为根据所述各个测试结果,确定所述车损识别模型针对所述测试样本集的评估结果。
根据第五方面,提供了一种计算机可读存储介质,其上存储有计算机程序,当所述计算机程序在计算机中执行时,令计算机执行第一方面和第二方面的方法。
根据第六方面,提供了一种计算设备,包括存储器和处理器,其特征在于,所述存储器中存储有可执行代码,所述处理器执行所述可执行代码时,实现第一方面和第二方面的方法。
通过本说明书实施例提供的方法和装置,采用多个标注人员对同一测试图片进行标注而产生的多组标注数据,确定出该测试图片中的显著损伤对象和非显著损伤对象。在为了评估车损识别模型而比对模型输出的预测损伤对象与标注数据时,对于显著损伤对象和非显著损伤对象进行区分,采用不同的比对方式,基于预测损伤对象分别与显著损伤对象和非显著损伤对象的关系,确定车损识别模型对测试样本的测试结果。由于对 显著损伤对象和非显著损伤对象进行了区分,使得在评估车损识别模型时,对这两种标注对象给予不同的关注和不同的衡量标准,从而避免由于非显著损伤对象的模糊性和不确定性而带来的评估上的争议和噪声,为模型优化找到更好的优化目标。
车体的损伤鉴别具有一定的专业性,车身的一些反光污渍与损伤容易混淆,容易引起标注人员的误标注或漏标注,比如车灯轻微损伤,由于车灯结构复杂,玻璃材质容易反光等因素,需要具有很强视觉能力和经验的人才能进行更为准确的标注。该方法可以有效得区分明确错误和不明确情况,帮助测试模型对明确错误的优化能力,并能通过在模型中增加明确错误权重,减少不明确情况权重,使模型对明确错误具有更好的区分性。该方法同样适用于其他需要专业知识和较高视觉能力的场景的评测和优化,比如医疗疾病的标注。
为了更清楚地说明本申请实施例的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它的附图。
图1示出根据一个实施例的车损图片的例子;
图2示出根据一个实施例的评估车损识别模型的方法流程图;
图3a示出在一个例子中的第1组标注数据;
图3b示出在一个例子中的第2组标注数据;
图3c示出在一个例子中的第3组标注数据;
图4示出显著损伤对象集合,非显著损伤对象集合,以及预测损伤对象集合的关系;
图5示出根据一个实施例的评估车损识别模型的方法;
图6示出根据一个实施例的评估装置的示意性框图;
图7示出根据一个实施例的评估装置的示意性框图。
下面结合附图,对本说明书提供的方案进行描述。
首先描述实施例方案的总体构思。该总体构思源于申请人对人视觉能力的分析和研究。
申请人经过观察和研究,认为人的视觉能力可以分为正常普通视觉能力和超视觉能力。正常普通视觉能力可以对于显著的对象进行准确识别,而超视觉能力会在正常普通视觉能力基础之上,观察和识别出非显著的对象。
在为了对车辆进行定损而进行车辆损伤识别的场景中,正常普通视觉能力的人可以注意到并观察到一般显著的损伤,而不显著的损伤则需要超视觉能力才能注意到,并且能够与干扰极强的反光纹理污渍等情况区分开来。
图1示出根据一个实施例的车损图片的例子。在该例子中,正常普通视觉能力的人都可以观察到,保险杠和翼子板上有两处划擦损伤,因此该损伤为显著损伤。然而,车辆左前大灯左下侧一点不透明的刮擦损伤,以及钢圈上的一点刮擦损伤则没有那么显著,只有视觉能力强,仔细观察每个角落的人才能观察到。
如本领域技术人员所知,机器学习模型需要基于大量的标注数据进行学习,才能实现良好的预测性能。作为机器学习模型的一种,车损识别模型则需要大量的车损标注数据作为训练样本进行训练,才能对未知图片进行车损识别。这些标注数据就是通过标注人员对类似于图1的车损图片进行标注而产生。
通常来说,为车损图片进行标注的标注人员是经过长期的视觉训练的专业人员。这些人员中会有一部分具有较强的视觉观察能力,能够观察并标注出部分非显著的损伤,尤其是通过专项的训练之后,可以对灯或小部件进行专业的标注。对于标注人员整体来说,显著损伤的标注都更为容易,标注基本不会有遗漏,但是非显著损伤的标注,则会因人而异,有所偏差。基于这样标注产生的训练样本进行车损识别模型的训练,模型对显著损伤会覆盖得很好,同时还会具备一定的超视觉能力才能观察的感知能力,因此模型的识别能力与显著损伤有较大的交集,与非显著损伤有一部分交集。
在评价车损识别模型的识别准确性和识别能力时,一种方案是将车损识别模型所识别出的损伤与保险公司人工确定出的定损单进行比对。然而,这样的方式常常带来很大争议。一种情况是,定损单中包含了一处车损识别模型没有召回的损伤,而该处损伤并不显著,这时,就会对模型的召回能力产生争议,因为有一些损伤由于视觉能力差异, 不能确定。另一种情况是,模型识别出了一些定损单中没有包含的非显著损伤,这时容易对模型的精度问题产生争议,因为有可能此处确实有损伤,而定损单没有反映出来。因此,直接比对模型识别结果和定损单结果的一致性时,会产生争议。
基于以上的观察、分析和研究,申请人提出一种方案,对车损图片中的显著损伤对象和非显著损伤对象进行区分。在评价车损识别模型的识别性能时,对于显著损伤对象和非显著损伤对象进行不同的判断标准,从而更有效地对损伤识别模型进行评估,避免不必要的争议。
下面首先描述,如何评估训练得到的车损识别模型对于单个测试样本,即单张车损图片,的识别效果。
图2示出根据一个实施例的评估车损识别模型的方法流程图。该方法可以由任何具有计算、处理能力的装置、设备、计算平台、计算集群来执行。如图2所示,该方法至少包括以下步骤:步骤21,获取第一测试样本,该第一测试样本对应第一车损图片以及多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损伤对象进行标注而产生;步骤22,确定所述多组标注数据的交集和并集,根据所述交集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;步骤23,将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;步骤24,根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。下面具体描述以上各个步骤的执行方式和执行过程。
首先,在步骤21,获取单个测试样本。为了描述的简单,将该测试样本称为第一测试样本。该第一测试样本对应单张车损图片,以下称为第一车损图片,以及针对该车损图片的多组标注数据,其中多组标注数据至少基于多个标注人员对上述第一车损图片中的损伤对象进行标注而产生。
具体而言,对于一张原始车损图片,可以将其分发给多个标注人员,使其分别对其中的车辆损伤对象进行标注,由此产生多组标注数据。该多组标注数据连同车损图片,共同构成一个测试样本。
在一个实施例中,为了整合不同视觉能力的标注人员的标注结果,将车损图片分发给不同标注能力的人员。在实际操作中,会根据标注人员的视觉观察能力的强弱,可 选的也会结合培训时间长短等因素,对其标注能力进行分级。通常,更高的标注能力等级意味着更强的视觉观察能力。在预先为标注人员进行分级的情况下,可以将车损图片分发给具有不同标注能力等级的标注人员,由此产生多组标注数据。
在一个具体例子中,可以将车损图片分发给具有第一标注能力等级的标注人员进行标注,从而产生第一标注数据;还将同样的车损图片分发给具有第二标注能力等级的标注人员进行标注,从而产生第二标注数据,其中第二标注能力等级高于第一标注能力等级。如此,多组标注数据至少包括,由不同标注能力等级(第一和第二能力等级)的标注人员分别产生的第一标注数据和第二标注数据。
更具体的,在一个实施例中,上述第一标注能力等级对应于普通视觉能力,第二标注能力等级对应于超视觉能力。由此,多组标注数据中包括,由普通视觉能力的标注人员对车损图片进行标注所产生的数据,以及由超视觉能力的标注人员对上述图片进行标注所产生的数据。在其他实施例中,还可以对标注能力等级进行更细致的划分,例如分为三个等级,甚至四个等级,并使得上述多组标注数据来自于不同标注能力等级的标注人员。
在一个实施例中,对于每组标注数据,均是由标注人员进行标注之后,再由核查人员进行核查确认而产生。如此,避免个别标注人员的错误标注对整个测试的影响。
具体地,在一个例子中,将图1所示的车损图片作为原图,分发给3个标注人员进行损伤对象的标注,得到3组标注数据。图3a-3c分别示出这3组标注数据。可以看到,由于不同标注人员视觉能力的差异,3组标注数据不尽相同。图3a中包含了2处标注出来的损伤对象,分别记为A1和A2;图3b包含的标注损伤对象为A1、A2和新增的A3;而图3c则包含了5处标注出来的损伤对象,分别为A1、A2、A4、A5和A6。这3组标注数据,连同图1所示的车损原图,就可以作为一个测试样本。
接下来,在步骤22,确定多组标注数据的交集和并集,根据交集确定第一测试样本中的显著损伤对象集合;根据交集与并集之间的差异,确定第一测试样本中的非显著损伤对象集合。
如前所述,多组标注数据来自于多个不同的标注人员,标注出来的损伤对象不尽相同。因此,在步骤22,根据多组标注数据之间的异同,确定哪些损伤对象是显著损伤对象,哪些是非显著损伤对象。
具体而言,如果一处损伤,所有标注人员均可观察到并将其标注出来,则该处损 伤应该确定为显著损伤对象。这对应于多组标注数据的交集。因此,将上述多组标注数据的交集中的损伤对象确定为显著损伤对象。相应地,将除显著损伤对象之外的其他对象,确定为非显著损伤对象,也就是,将多组标注数据的并集与上述交集之间的差异,确定为非显著损伤对象。
结合图3a-3c的例子进行描述。图3a包含的损伤对象集合为{A1,A2},图3b包含的损伤对象集合为{A1,A2,A3},图3c包含的损伤对象集合为{A1,A2,A4,A5,A6}。对于这3组标注数据,其交集为{A1,A2},其并集为{A1,A2,A3,A4,A5,A6}。因此,在一个实施例中,根据交集确定出,针对图1的车损图片,显著损伤对象集合为{A1,A2},非显著损伤对象集合为{A3,A4,A5,A6}。
为了对车损识别模型进行评估,接着,在步骤23,将上述车损图片输入预先训练的车损识别模型,得到该车损识别模型针对该测试样本预测的损伤对象所构成的预测损伤对象集合。
上述车损识别模型可以基于各种机器学习算法,采用各种神经网络结构而实现,在此不做限定。车损识别模型基于训练样本集而训练,训练样本集包含大量的训练样本,每个训练样本对应一张车损图片,以及针对该车损图片的标注数据。以标注数据作为样本标签,可以对车损识别模型进行训练。在训练完成之后,将待识别图片输入车损识别模型,模型就会输出针对该图片的识别结果,又称为预测结果,其中包含所识别或所预测的各个损伤对象。
对于前述步骤中作为测试样本的第一车损图片,将其输入已经训练好的车损识别模型,模型就会输出针对该第一车损图片预测的各个损伤对象。将所预测的损伤对象构成的集合称为预测损伤对象集合。
然后,在步骤24,根据步骤23得到的预测损伤对象集合与步骤22得到的显著损伤对象集合和非显著损伤对象集合的关系,确定车损识别模型对第一测试样本的测试结果。
图4示出显著损伤对象集合,非显著损伤对象集合,以及预测损伤对象集合三者的关系。如图4所示,圆圈101表示显著损伤对象集合,对应于多组标注数据的交集。圆圈102对应于前述多组标注数据的并集,表示标注出的所有损伤对象。因此,圆圈102完全地包含圆圈101,圆圈101完全落入圆圈102之中。圆圈102和圆圈101之间的部分,也就是多组标注数据的并集与交集之间的差异部分,表示非显著损伤对象,这部分 损伤对象对应于模棱两可的损伤,或者只有超视觉能力的人才可以识别出的损伤。
根据本说明书实施例的构思,在理想的情况下,预测损伤对象集合103’应位于圆圈101和圆圈102之间,如图4中虚线所示。也就是,预测损伤对象集合103’完全包含显著损伤对象集合101,但不超出所有损伤对象的集合102,包含部分的非显著损伤对象。此时,可以认为,预测损伤对象集合103’结果正确,车损识别模型针对当前测试样本的测试结果较好。
然而,实际上,也会出现预测损伤对象集合103与理想情况出现偏差的情况,如图中加粗实线所示。此时,认为预测损伤对象集合103结果有误,车损识别模型针对当前测试样本的测试结果不够理想。
下面描述基于图4所示的关系,在步骤24中确定车损识别模型的测试结果的具体执行方式。
在一种实施方案中,确定车损识别模型针对单个样本的测试结果,即确定车损识别模型针对该单个样本,预测是否正确。相应的,测试结果包括,预测错误或预测正确的结论。
如前所述,在理想情况下,预测损伤对象集合应完全包含显著损伤对象集合,这意味着,要求车损识别模型预测出所有的显著损伤对象。因此,在一个实施例中,定义预测错误的一种错误类型,称为第一类错误。该第一类错误对应这样的情况,即,车损识别模型未能识别出或预测出某处显著损伤对象。由于显著损伤对象是普通视觉能力的人都会观察到的损伤对象,在车损识别模型的识别结果出现上述第一类错误的情况下,可以将车损识别模型针对当前的第一测试样本的测试结果确定为预测错误。
为此,在具体执行方式中,可以在步骤24中判断显著损伤对象集合是否为预测损伤对象集合的子集,即判断,预测损伤对象集合是否完全包含显著损伤对象集合;如果上述判断为否,则确定车损识别模型的测试结果为预测错误。
在图3a-图3c的例子中,如前所述,显著损伤对象集合为{A1,A2}。如果车损识别模型输出的预测损伤对象集合未能完全包含{A1,A2},则认为,针对该测试样本的测试结果为预测错误。
此外,如前所述,还希望预测损伤对象集合不超出所有损伤对象的集合。因此,在一个实施例中,定义预测错误的另一种错误类型,称为第二类错误。该第二类错误对应这样的情况,即,车损识别模型识别出的损伤,超出了标注的所有损伤对象范围。由 于该所有损伤对象范围(对应于图4中的圆圈102)包含了显著损伤对象和非显著损伤对象,或者说包含了所有可能的损伤对象,因此在车损识别模型的识别结果出现上述第二类错误的情况下,可以将车损识别模型针对当前的第一测试样本的测试结果确定为预测错误。
为此,在步骤24的具体执行方式中,对于预测损伤对象集合中任意的损伤对象,下文称为第一损伤对象,如果该第一损伤对象不属于显著损伤对象集合和非显著损伤对象集合中的任一个,则确定车损识别模型对当前测试样本的测试结果为预测错误。
在图3a-图3c的例子中,如前所述,显著损伤对象集合为{A1,A2},非显著损伤对象集合为{A3,A4,A5,A6},其并集为{A1,A2,A3,A4,A5,A6}。如果车损识别模型输出的预测损伤对象集合中包含某处损伤B1,不属于显著损伤对象集合和非显著损伤对象集合中的任一个,或者说不属于并集{A1,A2,A3,A4,A5,A6},则认为,车损识别模型召回了错误的对象,针对该测试样本的测试结果为预测错误。
与上述预测错误相对的,可以对应确定出预测正确的测试结果。在一个实施例中,如果预测损伤对象集合不存在上述第一类错误,也不存在上述第二类错误,则将车损识别模型针对当前测试样本的测试结果确定为预测正确。
以上,根据预测损伤对象集合与显著损伤对象集合和非显著损伤对象集合的关系,将车损识别模型针对当前测试样本的测试结果划分为预测错误或预测正确。
与此相对的,根据另一种实施方案,采用测试分数来表征车损识别模型针对单个测试样本的测试结果。该测试分数可以包括单样本的正向分数,正向分数用于表示针对该单样本的预测正确率,正确个数,正确性等等;测试分数也可以包括单样本的负向分数,负向分数例如表示针对该单样本的预测错误率,错误个数,等等。
在一个实施例中,可以确定预测损伤对象集合中所包含的、属于显著损伤对象集合的损伤对象的数目,称为第一数目,记为N1;还确定预测损伤对象集合中所包含的、属于非显著损伤对象集合的损伤对象的数目,称为第二数目,记为N2。然后,至少根据第一数目N1和第二数目N2,确定车损识别模型针对当前测试样本的单样本正向分数。
例如,在一个例子中,确定第一数目N1和第二数目N2之和作为预测正确的损伤对象数目N,用该数目N除以预测损伤对象集合中元素总数目M,将N/M作为上述正向分数,以刻画预测正确的损伤对象的占比。
又例如,在另一个例子中,确定第一数目N1与显著损伤集合中元素数目S1的比 例R1,并确定第二数目N2与非显著损伤集合中元素数目S2的比例R2,对R1和R2求平均,得到单样本正确率R作为上述正向分数,即:
R=(R1+R2)/2=(N1/S1+N2/S2)/2
仍然结合图3a-3c的例子进行描述。假定预测损伤对象集合P为{A1,A3,A4,A5,B1,B2},那么P中属于显著损伤对象集合的元素为A1,N1=1,根据之前的描述,显著损伤对象集合中元素数目S1=2;P中属于非显著损伤对象集合的元素为A3,A4和A5,因此N2=3,而非显著损伤对象集合中元素数目S2=4。于是,正向分数可以确定为R=(1/2+3/4)/2=0.625。
进一步的,在又一例子中,在对第一比例R1和第二比例R2求和时,为两者赋予不同的权重,即R=w1*R1+w2*R2。由于对显著损伤对象进行正确预测的重要性要远远高于非显著损伤对象,因此可以使得w1大于w2,甚至w1可以远大于w2。例如,令w1=0.8,w2=0.2。在一个实施例中,对于以上的示例,正向分数可以确定为R=0.8*0.5+0.2*0.75=0.55。
在一个实施例中,考虑上述第一类错误和第二类错误的负面影响,确定车损识别模型针对单个测试样本的负向分数。负向分数NS可以通过多种方式确定。
在一个实施例中,对于第一类错误和第二类错误分别赋予一定的分值,将负面分数NS确定为上述分值的求和,即:
NS=a*T1+b*T2
其中,a为第一类错误的分值,在出现第一类错误的情况下,T1=1,否则T1=0;b为第二类错误的分值,在出现第二类错误的情况下,T2=1,否则T2=0。
在另一个实施例中,还确定预测错误的损伤对象的数目。具体的,在一个例子中,确定包含在显著损伤对象集合中、但不包含在预测损伤对象集合中的损伤对象的数目,称为第三数目,记为N3,以及确定预测损伤对象集合中所包含的、不属于显著损伤对象集合也不属于非显著损伤对象集合的损伤对象的数目,称为第四数目,记为N4。以上第三数目和第四数目分别对应于上述第一类错误和第二类错误的情况,都可以认为是预测错误的数目。于是,可以根据第三数目和第四数目,确定车损识别模型针对当前测试样本的负向分数NS。
具体的,在一个例子中,可以将负向分数确定为:
NS=w3*N3+w4*N4,
其中,w3和w4为权重因子。
仍然结合预测损伤对象集合P为{A1,A3,A4,A5,B1,B2}的例子进行描述。针对该例子,可以确定出,显著损伤对象集合中包含的损伤对象A2没有包含在预测损伤对象集合中,因此,N3=1;另外,预测损伤对象集合中包含了损伤对象B1,B2,该损伤对象不属于显著损伤对象集合也不属于非显著损伤对象集合,因此,N4=2。
在一个例子中,假定负向分数NS=w3*N3+w4*N4,其中w3=0.6,w4=0.4,那么针对该样本的负向分数为NS=1*0.6+2*0.4=1.4。
通过以上方式,基于上述第一类错误/第二类错误的考虑,为测试结果给出负向打分。
根据一种实施方案,还可以将上述正向分数和负向分数进行综合,得到针对当前测试样本的单样本总分数。例如,对于正向分数PS和负向分数NS,可以将车损识别模型针对当前测试样本的单样本总分数S确定为:S=PS-NS。
如此,通过多种方式,根据预测损伤对象集合与显著损伤对象集合和非显著损伤对象集合的关系,评估车损识别模型针对当前测试样本的单样本测试结果。
在评估单个样本的测试结果的基础上,可以采用类似的方式,评估车损识别模型对多个测试样本构成的测试样本集的识别效果。
图5示出根据一个实施例的评估车损识别模型的方法,该方法用于评估车损识别模型对测试样本集中的多个测试样本的整体识别效果。如图5所示,首先在步骤51,获取测试样本集,其中包括多个测试样本。每个测试样本对应一个车损图片,以及多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生。多组标注数据和多个标注人员的描述可以参见对前述步骤21的描述,不再赘述。
接着,在步骤52,对于各个测试样本,执行图2所示的方法,从而确定出车损识别模型针对各个测试样本的各个测试结果。换而言之,对于测试样本集中的每个测试样本,将其作为图2方法中的第一测试样本,执行图2所示的方法,于是可以得到针对该测试样本的测试结果。通过对每个测试样本均执行上述方法,可以得到针对测试样本集中各个测试样本的各个测试结果。
然后,在步骤53,根据上述各个测试结果,确定车损识别模型针对测试样本集的评估结果。
在一个实施例中,如前所述,各个测试结果可以包括预测正确或预测错误的结果。在这样的情况下,确定车损识别模型针对测试样本集的评估结果可以包括,确定各个测试结果中预测正确的比例,将该比例作为评估结果。相应的,在该比例高于一定阈值,例如80%,的情况下,可以认为车损识别模型的识别效果满足要求,测试通过。
在另一实施例中,各个测试结果可以包括单样本测试分数。在这样的情况下,确定车损识别模型针对测试样本集的评估结果可以包括,根据各个测试结果中包括的单样本测试分数,确定针对测试样本集的总样本分数,将该分数作为评估结果。
更具体的,可以计算各个单样本测试分数的和值,平均值等,以此确定总样本分数。根据测试分数的具体含义以及总样本分数的计算方式,可以设置相应分数阈值,在总样本分数满足分数阈值的情况下,认为车损识别模型的识别效果满足要求。例如,在一个具体例子中,单样本测试分数为负向分数,总样本分数为各个单样本分数的平均值。在这样的情况下,如果总样本分数小于预设的分数阈值,则认为测试通过。
通过以上的方式,有效地评估车损识别模型对整个测试样本集的识别效果。
回顾车损识别模型的评估过程,在该过程中,通过多个标注人员对同一测试图片进行标注而产生的多组标注数据,确定出该测试图片中的显著损伤对象和非显著损伤对象。在为了评估车损识别模型而比对模型输出的预测损伤对象与标注数据时,对于显著损伤对象和非显著损伤对象进行区分,采用不同的比对方式,基于预测损伤对象分别与显著损伤对象和非显著损伤对象的关系,确定车损识别模型对测试样本的测试结果。由于对显著损伤对象和非显著损伤对象进行了区分,使得在评估车损识别模型时,对这两种标注对象给予不同的关注和不同的衡量标准,从而避免由于非显著损伤对象的模糊性和不确定性而带来的评估上的争议和噪声。
根据另一方面的实施例,还提供一种评估车损识别模型的装置。图6示出根据一个实施例的评估装置的示意性框图。该评估装置可以部署在任何具有计算、处理能力的装置、设备、计算平台、计算集群中,用于评估车损识别模型对单个测试样本的识别结果。如图6所示,该评估装置600包括:
样本获取单元61,配置为获取第一测试样本,该第一测试样本对应第一车损图片以及多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损伤对象进行标注而产生;
标注集合确定单元62,配置为确定所述多组标注数据的交集和并集,根据所述交 集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;
预测集合确定单元63,配置为将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;
测试结果确定单元64,配置为根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。
在一个实施例中,所述多组标注数据至少包括,第一标注数据和第二标注数据,其中第一标注数据由具有第一标注能力等级的标注人员标注产生,第二标注数据由具有第二标注能力等级的标注人员标注产生,所述第二标注能力等级高于所述第一标注能力等级。
根据一种实施方式,多组标注数据中每组标注数据通过标注人员进行标注,以及核查人员进行核查而产生。
在一种实施方案中,测试结果包括预测错误或预测正确的结果。
进一步的,在一个实施例中,所述测试结果确定单元64配置为:
判断所述显著损伤对象集合是否为所述预测损伤对象集合的子集;
如果不是其子集,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
在另一实施例中,所述测试结果确定单元64配置为:
对于所述预测损伤对象集合中任意的第一损伤对象,如果该第一损伤对象不属于所述显著损伤对象集合和所述非显著损伤对象集合中的任一个,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
在另一种实施方案中,测试结果包括单样本测试分数。
进一步的,在一个实施例中,所述测试结果确定单元64配置为:
确定所述预测损伤对象集合中所包含的、属于所述显著损伤对象集合的损伤对象的第一数目;
确定所述预测损伤对象集合中所包含的、属于所述非显著损伤对象集合的损伤对象的第二数目;
至少根据所述第一数目和第二数目,确定所述车损识别模型针对所述第一测试样本的单样本正向分数。
在另一实施例中,所述测试结果确定单元64配置为:
确定包含在所述显著损伤对象集合中、但不包含在所述预测损伤对象集合中的损伤对象的第三数目;
确定所述预测损伤对象集合中所包含的、不属于所述显著损伤对象集合也不属于所述非显著损伤对象集合的损伤对象的第四数目;
根据所述第三数目和第四数目,确定所述车损识别模型针对所述第一测试样本的单样本负向分数。
根据又一方面的实施例,还提供另一种评估车损识别模型的装置。图7示出根据一个实施例的评估装置的示意性框图。该评估装置700用于评估车损识别模型对多个测试样本构成的测试样本集的识别结果。如图7所示,该评估装置700包括:
样本集获取单元71,配置为获取测试样本集,其中包括多个测试样本,所述多个测试样本对应多个车损图片,每个车损图片对应具有多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生;
测试结果获取单元72,配置为对于各个测试样本,利用图6的装置600,确定出所述车损识别模型针对各个测试样本的各个测试结果:
评估结果确定单元73,配置为根据所述各个测试结果,确定所述车损识别模型针对所述测试样本集的评估结果。
在一个实施例中,其中所述各个测试结果包括预测正确或预测错误的结果;相应的,所述评估结果确定单元73配置为,确定所述各个测试结果中预测正确的比例,作为所述评估结果。
在另一实施例中,其中所述各个测试结果包括单样本测试分数;相应的,所述评估结果确定单元73配置为,根据所述各个测试结果中包括的单样本测试分数,确定针对所述测试样本集的总样本分数,将其作为所述评估结果。
根据另一方面的实施例,还提供一种计算机可读存储介质,其上存储有计算机程 序,当所述计算机程序在计算机中执行时,令计算机执行结合图2和图5所描述的方法。
根据再一方面的实施例,还提供一种计算设备,包括存储器和处理器,所述存储器中存储有可执行代码,所述处理器执行所述可执行代码时,实现结合图2和图5所述的方法。
本领域技术人员应该可以意识到,在上述一个或多个示例中,本申请所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时,可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。
以上所述的具体实施方式,对本申请的目的、技术方案和有益效果进行了进一步详细说明,所应理解的是,以上所述仅为本申请的具体实施方式而已,并不用于限定本申请的保护范围,凡在本申请的技术方案的基础之上,所做的任何修改、等同替换、改进等,均应包括在本申请的保护范围之内。
Claims (25)
- 一种评估车损识别模型的方法,包括:获取第一测试样本,该第一测试样本对应第一车损图片以及多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损伤对象进行标注而产生;确定所述多组标注数据的交集和并集,根据所述交集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。
- 根据权利要求1所述的方法,其中,所述多组标注数据至少包括,第一标注数据和第二标注数据,其中第一标注数据由具有第一标注能力等级的标注人员标注产生,第二标注数据由具有第二标注能力等级的标注人员标注产生,所述第二标注能力等级高于所述第一标注能力等级。
- 根据权利要求1所述的方法,其中,所述多组标注数据中每组标注数据通过标注人员进行标注,以及核查人员进行核查而产生。
- 根据权利要求1所述的方法,其中,所述测试结果包括,预测错误或预测正确。
- 根据权利要求4所述的方法,其中,确定所述车损识别模型对所述第一测试样本的测试结果,包括:判断所述显著损伤对象集合是否为所述预测损伤对象集合的子集;如果不是其子集,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
- 根据权利要求4或5所述的方法,其中,确定所述车损识别模型对所述第一测试样本的测试结果,包括:对于所述预测损伤对象集合中任意的第一损伤对象,如果该第一损伤对象不属于所述显著损伤对象集合和所述非显著损伤对象集合中的任一个,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
- 根据权利要求1所述的方法,其中,所述测试结果包括单样本测试分数。
- 根据权利要求7所述的方法,其中,确定所述车损识别模型对所述第一测试样本的测试结果,包括:确定所述预测损伤对象集合中所包含的、属于所述显著损伤对象集合的损伤对象的第一数目;确定所述预测损伤对象集合中所包含的、属于所述非显著损伤对象集合的损伤对象的第二数目;至少根据所述第一数目和第二数目,确定所述车损识别模型针对所述第一测试样本的单样本正向分数。
- 根据权利要求7或8所述的方法,其中,确定所述车损识别模型对所述第一测试样本的测试结果,包括:确定包含在所述显著损伤对象集合中、但不包含在所述预测损伤对象集合中的损伤对象的第三数目;确定所述预测损伤对象集合中所包含的、不属于所述显著损伤对象集合也不属于所述非显著损伤对象集合的损伤对象的第四数目;根据所述第三数目和第四数目,确定所述车损识别模型针对所述第一测试样本的单样本负向分数。
- 一种评估车损识别模型的方法,包括:获取测试样本集,其中包括多个测试样本,所述多个测试样本对应多个车损图片,每个车损图片对应具有多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生;对于各个测试样本,执行权利要求1所述的方法,从而确定出所述车损识别模型针对各个测试样本的各个测试结果:根据所述各个测试结果,确定所述车损识别模型针对所述测试样本集的评估结果。
- 根据权利要求10所述的方法,其中所述各个测试结果包括预测正确或预测错误的结果;所述确定所述车损识别模型针对所述测试样本集的评估结果包括,确定所述各个测试结果中预测正确的比例,作为所述评估结果。
- 根据权利要求10所述的方法,其中所述各个测试结果包括单样本的测试分数;所述确定所述车损识别模型针对所述测试样本集的评估结果包括,根据所述各个测试结果中包括的单样本的测试分数,确定针对所述测试样本集的总样本分数,将其作为所述评估结果。
- 一种评估车损识别模型的装置,包括:样本获取单元,配置为获取第一测试样本,该第一测试样本对应第一车损图片以及 多组标注数据,所述多组标注数据至少基于多个标注人员对所述第一车损图片中的损伤对象进行标注而产生;标注集合确定单元,配置为确定所述多组标注数据的交集和并集,根据所述交集确定所述第一测试样本中的显著损伤对象集合;并根据所述交集与所述并集之间的差异,确定所述第一测试样本中的非显著损伤对象集合;预测集合确定单元,配置为将所述第一车损图片输入预先训练的车损识别模型,得到所述车损识别模型针对该第一测试样本预测的损伤对象所构成的预测损伤对象集合;测试结果确定单元,配置为根据所述预测损伤对象集合与所述显著损伤对象集合和所述非显著损伤对象集合的关系,确定所述车损识别模型对所述第一测试样本的测试结果。
- 根据权利要求13所述的装置,其中,所述多组标注数据至少包括,第一标注数据和第二标注数据,其中第一标注数据由具有第一标注能力等级的标注人员标注产生,第二标注数据由具有第二标注能力等级的标注人员标注产生,所述第二标注能力等级高于所述第一标注能力等级。
- 根据权利要求13所述的装置,其中,所述多组标注数据中每组标注数据通过标注人员进行标注,以及核查人员进行核查而产生。
- 根据权利要求13所述的装置,其中,所述测试结果包括,预测错误或预测正确。
- 根据权利要求16所述的装置,其中,所述测试结果确定单元配置为:判断所述显著损伤对象集合是否为所述预测损伤对象集合的子集;如果不是其子集,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
- 根据权利要求16或17所述的装置,其中,所述测试结果确定单元配置为:对于所述预测损伤对象集合中任意的第一损伤对象,如果该第一损伤对象不属于所述显著损伤对象集合和所述非显著损伤对象集合中的任一个,则确定所述车损识别模型对所述第一测试样本的测试结果为预测错误。
- 根据权利要求13所述的装置,其中,所述测试结果包括单样本测试分数。
- 根据权利要求19所述的装置,其中,所述测试结果确定单元配置为:确定所述预测损伤对象集合中所包含的、属于所述显著损伤对象集合的损伤对象的第一数目;确定所述预测损伤对象集合中所包含的、属于所述非显著损伤对象集合的损伤对象的第二数目;至少根据所述第一数目和第二数目,确定所述车损识别模型针对所述第一测试样本的单样本正向分数。
- 根据权利要求19或20所述的装置,其中,所述测试结果确定单元配置为:确定包含在所述显著损伤对象集合中、但不包含在所述预测损伤对象集合中的损伤对象的第三数目;确定所述预测损伤对象集合中所包含的、不属于所述显著损伤对象集合也不属于所述非显著损伤对象集合的损伤对象的第四数目;根据所述第三数目和第四数目,确定所述车损识别模型针对所述第一测试样本的单样本负向分数。
- 一种评估车损识别模型的装置,包括:样本集获取单元,配置为获取测试样本集,其中包括多个测试样本,所述多个测试样本对应多个车损图片,每个车损图片对应具有多组标注数据,所述多组标注数据至少基于多个标注人员对该车损图片中的损伤对象进行标注而产生;测试结果获取单元,配置为对于各个测试样本,利用权利要求13所述的装置,确定出所述车损识别模型针对各个测试样本的各个测试结果:评估结果确定单元,配置为根据所述各个测试结果,确定所述车损识别模型针对所述测试样本集的评估结果。
- 根据权利要求22所述的装置,其中所述各个测试结果包括预测正确或预测错误的结果;所述评估结果确定单元配置为,确定所述各个测试结果中预测正确的比例,作为所述评估结果。
- 根据权利要求22所述的装置,其中所述各个测试结果包括单样本测试分数;所述评估结果确定单元配置为,根据所述各个测试结果中包括的单样本测试分数,确定针对所述测试样本集的总样本分数,将其作为所述评估结果。
- 一种计算设备,包括存储器和处理器,其特征在于,所述存储器中存储有可执行代码,所述处理器执行所述可执行代码时,实现权利要求1-12中任一项所述的方法。
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910002372.0A CN110009509B (zh) | 2019-01-02 | 2019-01-02 | 评估车损识别模型的方法及装置 |
CN201910002372.0 | 2019-01-02 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2020140684A1 true WO2020140684A1 (zh) | 2020-07-09 |
Family
ID=67165331
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2019/123344 WO2020140684A1 (zh) | 2019-01-02 | 2019-12-05 | 评估车损识别模型的方法及装置 |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN110009509B (zh) |
WO (1) | WO2020140684A1 (zh) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110009509B (zh) * | 2019-01-02 | 2021-02-19 | 创新先进技术有限公司 | 评估车损识别模型的方法及装置 |
CN110738576B (zh) * | 2019-10-14 | 2021-08-06 | 支付宝(杭州)信息技术有限公司 | 为受损车辆生成定损文件的方法及装置 |
CN112037235B (zh) * | 2020-08-27 | 2023-01-10 | 平安科技(深圳)有限公司 | 伤情图片自动化审核方法、装置、电子设备及存储介质 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012151198A1 (en) * | 2011-05-04 | 2012-11-08 | Google Inc. | Predictive analytical modeling accuracy assessment |
CN107194398A (zh) * | 2017-05-10 | 2017-09-22 | 平安科技(深圳)有限公司 | 车损部位的识别方法及*** |
CN107358596A (zh) * | 2017-04-11 | 2017-11-17 | 阿里巴巴集团控股有限公司 | 一种基于图像的车辆定损方法、装置、电子设备及*** |
CN108446618A (zh) * | 2018-03-09 | 2018-08-24 | 平安科技(深圳)有限公司 | 车辆定损方法、装置、电子设备及存储介质 |
CN108615071A (zh) * | 2018-05-10 | 2018-10-02 | 阿里巴巴集团控股有限公司 | 模型测试的方法及装置 |
WO2018191421A1 (en) * | 2017-04-11 | 2018-10-18 | Alibaba Group Holding Limited | Image-based vehicle damage determining method, apparatus, and electronic device |
CN110009509A (zh) * | 2019-01-02 | 2019-07-12 | 阿里巴巴集团控股有限公司 | 评估车损识别模型的方法及装置 |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105279772B (zh) * | 2015-10-23 | 2018-02-09 | 中国运载火箭技术研究院 | 一种红外序列图像的可跟踪性判别方法 |
CN107133628A (zh) * | 2016-02-26 | 2017-09-05 | 阿里巴巴集团控股有限公司 | 一种建立数据识别模型的方法及装置 |
CN108229543A (zh) * | 2017-12-22 | 2018-06-29 | 中国科学院深圳先进技术研究院 | 图像分类模型设计方法及装置 |
CN108734702A (zh) * | 2018-04-26 | 2018-11-02 | 平安科技(深圳)有限公司 | 车损判定方法、服务器及存储介质 |
CN109034188B (zh) * | 2018-06-15 | 2021-11-05 | 北京金山云网络技术有限公司 | 机器学习模型的获取方法、获取装置、设备及存储介质 |
-
2019
- 2019-01-02 CN CN201910002372.0A patent/CN110009509B/zh active Active
- 2019-12-05 WO PCT/CN2019/123344 patent/WO2020140684A1/zh active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012151198A1 (en) * | 2011-05-04 | 2012-11-08 | Google Inc. | Predictive analytical modeling accuracy assessment |
CN107358596A (zh) * | 2017-04-11 | 2017-11-17 | 阿里巴巴集团控股有限公司 | 一种基于图像的车辆定损方法、装置、电子设备及*** |
WO2018191421A1 (en) * | 2017-04-11 | 2018-10-18 | Alibaba Group Holding Limited | Image-based vehicle damage determining method, apparatus, and electronic device |
CN107194398A (zh) * | 2017-05-10 | 2017-09-22 | 平安科技(深圳)有限公司 | 车损部位的识别方法及*** |
CN108446618A (zh) * | 2018-03-09 | 2018-08-24 | 平安科技(深圳)有限公司 | 车辆定损方法、装置、电子设备及存储介质 |
CN108615071A (zh) * | 2018-05-10 | 2018-10-02 | 阿里巴巴集团控股有限公司 | 模型测试的方法及装置 |
CN110009509A (zh) * | 2019-01-02 | 2019-07-12 | 阿里巴巴集团控股有限公司 | 评估车损识别模型的方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN110009509A (zh) | 2019-07-12 |
CN110009509B (zh) | 2021-02-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020140684A1 (zh) | 评估车损识别模型的方法及装置 | |
CN104346372B (zh) | 用于评估预测模型的方法和装置 | |
TWI716012B (zh) | 樣本標註方法、裝置、儲存媒體和計算設備、損傷類別的識別方法及裝置 | |
CN108491486B (zh) | 模拟病人问诊对话方法、装置、终端设备及存储介质 | |
CN110569703B (zh) | 计算机执行的从图片中识别损伤的方法及装置 | |
WO2020238256A1 (zh) | 基于弱分割的损伤检测方法及装置 | |
CN105868524A (zh) | 用于医学图像集的自动基准真值生成 | |
CN113963304B (zh) | 基于时序-空间图的跨模态视频时序动作定位方法及*** | |
Bollaerts et al. | Estimating the population prevalence and force of infection directly from antibody titres | |
CN112101315A (zh) | 一种基于深度学习的运动评判指导方法和*** | |
CN113378804A (zh) | 一种自助采样的检测方法、装置、终端设备和存储介质 | |
KR102232880B1 (ko) | 인공지능 학습데이터 생성을 위한 이미지 또는 동영상 수집을 위한 크라우드소싱 기반 프로젝트의 검수자 평가 방법 | |
CN112101074A (zh) | 一种在线教育辅助评分方法和*** | |
CN112884060A (zh) | 图像标注方法、装置、电子设备及存储介质 | |
CN113139962A (zh) | 用于脊柱侧凸概率评估的***和方法 | |
CN113313359A (zh) | 一种影像标注诊断质量的评估方法及装置 | |
CN112115714B (zh) | 深度学习序列标注方法、装置及计算机可读存储介质 | |
CN116504408A (zh) | 一种医生模拟诊断方法、***、计算机设备和存储介质 | |
CN116153530A (zh) | 基于口腔扫描视频的正畸治疗监测方法及装置 | |
CN116052108A (zh) | 基于Transformer的交通场景小样本目标检测方法及装置 | |
CN116228660A (zh) | 胸部正片异常部位检测方法及装置 | |
CN112669973B (zh) | 基于大数据深度学习的疾病协同递进预测方法和机器人 | |
CN113450381B (zh) | 一种图像分割模型的准确度评价***及方法 | |
CN114140751A (zh) | 一种考场监控方法及*** | |
CN111459050B (zh) | 一种基于双网互联的智慧仿真型护理教学***及教学方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19907699 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19907699 Country of ref document: EP Kind code of ref document: A1 |