CN114116456A - Test case generation method, system and computer readable storage medium - Google Patents
Test case generation method, system and computer readable storage medium Download PDFInfo
- Publication number
- CN114116456A CN114116456A CN202111301946.8A CN202111301946A CN114116456A CN 114116456 A CN114116456 A CN 114116456A CN 202111301946 A CN202111301946 A CN 202111301946A CN 114116456 A CN114116456 A CN 114116456A
- Authority
- CN
- China
- Prior art keywords
- sample
- new
- test case
- ini
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a test case generation method, a test case generation system and a computer readable storage medium. The test case generation method comprises the following steps: s1: obtaining initial image sample x from existing test case of model to be testediniFor the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, and obtaining output y 'corresponding to each new sample x'; s2: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions: zi(x)＝a0+a1x1+…+amxm(ii) a S3: calculating unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x')|≤λ；S4: according to an unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmObtaining a sample x for new testout，xout＝(x1,x2,…，xm). According to the test case generation method and system provided by the embodiment of the invention, the generated new test case can effectively find the error of the model to be tested.
Description
Technical Field
The invention relates to the technical field of computer software, in particular to a test case generation method and system and a computer readable storage medium.
Background
The image classification problem is artificial intelligence, an important problem in the field of computer vision, and practical problems such as face recognition, article recognition and the like can be converted into the image classification problem. At present, the method for constructing a neural network image classification model by using the existing data set is an effective technical means for solving the problem of image classification, and is also the most widely applied main technical means.
Image classification can be described specifically as the following problem: for the classes numbered 1, 2, … …, n, an image x belonging to one of the classes is entered, which we wish to have the correct class number. In this scenario, we also call the classified images "sample" and the class number "label". The neural network image classification model can be regarded as a function f, and the function receives an m-dimensional vector and outputs an n-dimensional vector. Where each element of the input m-dimensional vector corresponds to a pixel value of the input sample and each element of the output n-dimensional vector corresponds to a likelihood score for the sample being classified into a class of samples. We use fi(x) An i-th dimension element representing the output of the neural network when receiving sample x, then for some i, if fi(x) Greater than all other elements in the output, we get the classification result i of the neural network, because then the probability score that the neural network considers the category of x to be i is the highest.
Model testing is an important way to ensure that a model can correctly and effectively complete its functions. The data used for testing is called a "test case," which is generally composed of two parts: (1) inputting a sample; (2) the correct output corresponding to the input. The general test procedure is as follows: we input samples to the model being tested and get the output of the model. If the model output is consistent with the correct output, the model is correct under the test case, otherwise, an error of the model is found. The purpose of the test is to effectively find out the possible errors of the model through the test case. Therefore, in the neural network image classification model test, one test case of the neural network image classification model test comprises one sample image and the correct label of the sample image. In the testing process, the sample is input into the model to be tested, and whether the output result of the model is consistent with the correct label or not is checked.
For the neural network image classification model test, the test case of the neural network image classification model test is derived from an original data set really acquired for solving the classification problem, one part of the data set is used for constructing the model, and the other part of the data set is used as the test case. In fact, the number of test cases derived from the raw data set is very limited, and the input space of the neural network model is huge, which means that the test cases derived from the raw data set far from covering the input samples that the neural network model may receive. In other words, test cases derived from the raw data set may not adequately discover the potential errors present in the model.
Therefore, the test case can be derived from manual construction on the other hand, namely, a new test case generated by a certain method based on the existing test case is used for expanding the test case set. A mainstream test case generation method is to add random noise in a certain disturbance range to an image sample of an existing test case to obtain a new sample. The method is a mainstream test case generation method, because the method simulates noise disturbance in a real scene, the new test case generated according to the method can potentially discover the phenomenon that some neural network image classification models are wrongly classified due to noise interference on samples.
However, this method of generating new test cases by adding random noise directly to the existing test case samples also has its disadvantages. Since this method only focuses on the sample itself and does not use the properties of the neural network image classification model under test at all, this results in that most of the outputs of the generated new test samples are correct. In other words, the newly generated test case cannot effectively find the potential error of the network to be tested, and thus the practical significance of the test is lost.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a test case generation method, system and computer-readable storage medium capable of generating a new test case for effectively discovering a model under test.
In one aspect, an embodiment of the present invention provides a test case generation method, which includes the following steps:
s1: obtaining initial image sample x from existing test case of model to be testedini，xiniIs a vector of m dimensions, xiniC, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
s2: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
s3: calculating unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
s4: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
As a further improvement of the above embodiment, in step S1, the initial image sample x is subjected toiniThe process of adding random noise includes the following sub-steps:
s11: for initial image sample xiniEach dimension x ofiCalculating pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
As a further improvement of the above embodiment, in step S3, the optimization problem is solved by a linear programming method:
minλ
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
As a further improvement of the above embodiment, in step S4, for the unknown parameter a having been obtained0,a1,…,amAffine functions of definite values, Zi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a Radix CodonopsisNumber aiWhen it is negative, take xi＝pi(ii) a X is to be1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
Another aspect of an embodiment of the present invention provides a test case generation system, which includes:
a disturbance sample set generation module: obtaining initial image sample x from existing test case of model to be testedini，xiniIs a vector of m dimensions, xiniC, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
an affine function generating module: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
the affine function generating module also calculates the unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
the new test sample generation module: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
As a further improvement of the above embodiment, the perturbed sample set generating module generates the initial image sample xiniThe process of adding random noise includes the following sub-steps:
s11: for initial image sample xiniEach dimension x ofiCalculating pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
As a further improvement of the above embodiment, the affine function generating module solves the optimization problem by a linear programming method:
minλ
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
As a further improvement of the above embodiment, the new test sample generation module is configured to obtain the unknown parameter a0,a1,…,amAffine function Z for determining valuesi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a When parameter aiWhen it is negative, take xi＝pi(ii) a X is to be1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
Yet another aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the test case generation method according to any of the above embodiments.
According to the test case generation method and system provided by the embodiment of the invention, the existing test case is utilized to construct an approximate model which has similar properties and a definite structure with the model to be tested, and the generated new test case is generated through analysis of the approximate model, so that the generated new test case can effectively find the errors of the model to be tested.
Drawings
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings. Like reference numerals refer to like parts throughout the drawings, and the drawings are not intended to be drawn to scale in actual dimensions, emphasis instead being placed upon illustrating the principles of the invention.
Fig. 1 is a schematic diagram of a test case generation method according to an embodiment of the present invention.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The basic idea of the embodiment of the invention is as follows:
by xiThe element representing the ith dimension of a vector x, for the output of the vector x in the model under testThe output vector y is similar. For an existing sample x, the correct classification result is c. At this time, it means that it outputs the c-th dimension element of y, i.e., ycOther dimension elements that should be larger than their output, i.e. yi-yc<0。
By zi(x) Denotes yi-ycThis value is a function of x if we can generate a z around the original sample xi(x) If the sample is as large as possible, the new sample is likely to make the model under test misclassify, that is, we obtain a valid new test case. However, since the neural network model under test is very complex in structure, z isi(x) It is difficult to express explicitly with indirect functions. In other words, directly to zi(x) Performing the analysis is difficult.
Therefore, the basic idea of the invention is to construct a simple model Z based on the existing test case sample xi(x) And through Zi(x) Effective approximation of z near x for the model under testi(x) A function. Then for this Z with simple structurei(x) Performing an analysis, calculating the maximum point thereof, i.e. obtaining Zi(x) As large a sample as possible, to obtain a new test case that is most likely to cause the model under test to be erroneous.
In order to implement the basic idea of the present invention, an embodiment of the present invention provides a test case generating method, which includes the following steps:
s1: obtaining initial image sample x from existing test case of model to be testedini，xiniIs a vector of m dimensions, xiniC, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
s2: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
s3: calculating unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
s4: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
In this embodiment, the model to be tested may be a neural network image classification model. In step S1, for an existing test case of the neural network image classification model to be tested, a random noise δ is added to the image sample x in the test case within a certain disturbance range to obtain a new sample x '═ x + δ, and this process is repeated K times to obtain a disturbance sample set with K new samples x'. Then, inputting each sample x 'in the newly obtained disturbance sample set into the model to be measured one by one to obtain corresponding output y'. The perturbation employed may be performed as follows: i.e. given a perturbation radius r for a sample pixel value, the added perturbation is a vector delta of the same dimension as the sample x, where-r ≦ deltaiR is less than or equal to r. However, it should be noted that the pixel value must be 0 ~ 255, so the pixel value x after perturbationi+δiThis constraint should also be received.
And adding random noise in a certain disturbance range on the original sample x to obtain a plurality of new samples and corresponding output of the model to be tested to form a plurality of pairs of input and output data (x ', y'). Through the data pairs, the behavior characteristics and properties of the model to be tested in a certain disturbance range of the image sample x of the original test case can be represented, so that an approximate model can be further constructed.
In steps S2 and S3, a simple model Z is constructed by using the pairs of data (x ', y') obtained in the previous stepi(x) And through Zi(x) Effective approximation of z near x for the model under testi(x) A function. The learning by model is to make the new model Zi(x) Z with respect to the original modeli(x) The similarity of (2) can be ensured. In other words, this step ensures Zi(x') + λ is ziAn approximate upper bound of (x'), and Zi(x') - λ is ziAn approximate lower bound of (x'). In step S3, a set of unknown parameters a is obtained by calculating the input/output data (x ', y') of a plurality of new samples x0,a1,…,amThe set of unknown parameters a0,a1,…,amCan make Zi(x') satisfies | Zi(x')–zi(x') | is less than or equal to lambda.
In a preferred embodiment, in step S1, the initial image sample x is samplediniThe process of adding random noise includes the following sub-steps:
s11: for initial image sample xiniEach dimension x ofiCalculating pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: assigning x 'to each dimension'iCombine to obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
The above steps S11-S13 are repeated K times, so that K new samples x' can be obtained. Then, inputting each new sample x ' into the model to be tested, obtaining corresponding output y ', obtaining K pairs of data (x ', y '), and the set of the new samples x ' is a disturbance sample set.
In a preferred embodiment, in step S3, the optimization problem is solved by a linear programming method:
minλ
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
The above embodiment converts the problem of solving unknown parameters into an optimization problem, and adopts a linear programming method to solve the problem. The purpose of this step is to utilize Zi(x') to approximate zi(x') behavior in the perturbation range, so we want the distance between them as small as possible. The optimization objective of the constructed optimization problem is therefore to minimize the distance λ between them, i.e. to minimize Z in all new samples xi(x’)–ziThe difference of (x') is minimal.
In a preferred embodiment, in step S4, for unknown parameter a having been obtained0,a1,…,amAffine function of determined value: zi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a When parameter aiWhen it is negative, take xi＝pi(ii) a Determining x according to the above-described manner1,x2,…，xmValue of (a), x to be determined1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
Due to ZiIs an affine function of the form:
Zi(x)＝a0+a1 x1+…+amxm，
thus ZiAbout each xiAre all monotonic: when parameter aiIs a positive timing, xiThe larger, ZiThe larger; when parameter aiWhen it is negative, xiThe smaller, ZiThe smaller. Thus, for the perturbation range [ p ]1,q1]×…×[pm,qm]We can generate a Z byiMaximum point: when parameter aiWhen it is positive, take xi＝qiTaking the maximum value of the disturbance interval; when parameter aiWhen it is negative, take xi＝piI.e. taking the minimum value of its perturbation interval.
An embodiment of the present invention further provides a test case generation system, which includes:
a disturbance sample set generation module: obtaining initial image sample x from existing test case of model to be testedini，xiniIs a vector of m dimensions, xiniC, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
an affine function generating module: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
the affine function generating module also calculates the unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
the new test sample generation module: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
In a preferred embodiment, the perturbation sample set generation module generates initial image samples xiniThe process of adding random noise includes the following sub-steps:
s11: for initial image sample xiniEach dimension x ofiCalculating pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
In a preferred embodiment, the affine function generating module solves the optimization problem by a linear programming method:
minλ
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
Excellence inIn an optional embodiment, the new test sample generation module obtains the unknown parameter a0,a1,…,amAffine function Z for determining valuesi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a When parameter aiWhen it is negative, take xi＝pi(ii) a X is to be1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
The specific working principles of the disturbance sample set generation module, the affine function generation module and the new test sample generation module are consistent with the description of the test case generation method, and are not described herein again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the test case generation method according to any of the above embodiments are implemented.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system, various types of application software, and the like installed in the computer device. Further, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
The processor may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is configured to run a program code stored in a computer-readable storage medium or process data to implement the test case generation method of the above embodiment.
According to the test case generation method and system provided by the embodiment of the invention, the existing test case is utilized to construct an approximate model which has similar properties and a definite structure with the model to be tested, and the generated new test case is generated through analysis of the approximate model, so that the generated new test case can effectively find the errors of the model to be tested. According to the technical scheme, the test cases which enable the neural network image classification model to generate errors can be generated more effectively based on model abstraction, on one hand, a test means which can find potential errors of the neural network more effectively is provided, on the other hand, the test cases can be used for further training of the neural network, and therefore the performance of the neural network is improved.
The above examples only express specific embodiments of the invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (9)
1. A test case generation method is characterized by comprising the following steps:
s1: obtaining initial image sample x from existing test case of model to be testedini，xiniIs a vector of m dimensions, xiniIs correctly classifiedThe result is c, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
s2: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
s3: calculating unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
s4: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
2. The test case generation method of claim 1, wherein in step S1, for an initial image sample xiniThe process of adding random noise includes the following sub-steps:
s11: for initial image sample xiniEach dimension x ofiMeter for measuringCalculation of pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
3. The test case generation method according to claim 1 or 2, wherein in step S3, the optimization problem is solved by a linear programming method:
minλ
s.t.
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
4. The test case generation method according to claim 2, wherein in step S4, for the unknown parameter a having been obtained0,a1,…,amAffine functions of definite values, Zi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a When parameter aiWhen it is negative, take xi＝pi(ii) a X is to be1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
5. A test case generation system, comprising:
a disturbance sample set generation module: obtaining initial image sample x from existing test case of model to be testedini，xiniIs one m-dimensional directionAmount, xiniC, for the initial image sample xiniAdding random noise for multiple times to generate a disturbance sample set, wherein the disturbance sample set comprises a plurality of initial image samples xiniOn the basis, adding new samples x 'of random noise, and inputting each new sample x' of the disturbance sample set into the model to be tested to obtain output y 'corresponding to each new sample x';
an affine function generating module: for the other dimensions i with the division dimension of c of each new sample x', respectively establishing affine functions:
Zi(x)＝a0+a1 x1+…+amxm，
wherein a is0,a1,…,amFor unknown parameters to be determined, Zi(x) Is zi(x) The affine function of (a) is,
zi(x)＝yi–yc，
y is the output of the sample x in the model to be measured, yiAnd ycThe ith and the c-th dimension elements which are y respectively;
the affine function generating module also calculates the unknown parameters a according to the plurality of new samples x' and the corresponding outputs y0,a1,…,amSo that an affine function Zi(x) Satisfy | Zi(x')–zi(x') less than or equal to lambda, wherein lambda is a preset error;
the new test sample generation module: according to the calculated unknown parameter a0,a1,…,amCalculating Z isi(x) Obtaining x of the maximum point1,x2,…，xmWill make Zi(x) Obtaining x of the maximum point1,x2,…，xmThe formed vector is sample x for new testout，xout＝(x1,x2,…，xm)。
6. The test case generation system of claim 5, wherein the perturbation sample set generation module generates an initial image sample xiniBy adding random noiseThe process comprises the following substeps:
s11: for initial image sample xiniEach dimension x ofiCalculating pi＝min(0,xi-r) and qi＝max(255,xi+r)；
S12: for each dimension, randomly obtaining an interval [ p ]i，qi]Is assigned to x'i；
S13: obtain a new sample x ', x ═ x'1,x'2,…,x'm)。
7. The test case generation system of claim 5, wherein the affine function generation module solves an optimization problem by a linear programming method:
minλ
s.t.
the unknown parameter a which minimizes lambda in all the new samples x0,a1,…,amAs the unknown parameter a0,a1,…,amA determined value.
8. The test case generation method of claim 6, wherein the new test case sample generation module is configured to obtain an unknown parameter a0,a1,…,amAffine function Z for determining valuesi(x)＝a0+a1 x1+…+amxmWhen the parameter aiWhen it is positive, take xi＝qi(ii) a When parameter aiWhen it is negative, take xi＝pi(ii) a X is to be1,x2,…，xmAs sample x for the new testoutCorresponding to the value of each dimension, thereby forming a sample x for new testout。
9. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the test case generation method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111301946.8A CN114116456A (en) | 2021-11-04 | 2021-11-04 | Test case generation method, system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111301946.8A CN114116456A (en) | 2021-11-04 | 2021-11-04 | Test case generation method, system and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114116456A true CN114116456A (en) | 2022-03-01 |
Family
ID=80380726
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111301946.8A Pending CN114116456A (en) | 2021-11-04 | 2021-11-04 | Test case generation method, system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114116456A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292155A (en) * | 2022-06-22 | 2022-11-04 | 广州汽车集团股份有限公司 | Test case generation method and device and vehicle |
-
2021
- 2021-11-04 CN CN202111301946.8A patent/CN114116456A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115292155A (en) * | 2022-06-22 | 2022-11-04 | 广州汽车集团股份有限公司 | Test case generation method and device and vehicle |
| CN115292155B (en) * | 2022-06-22 | 2024-01-16 | 广州汽车集团股份有限公司 | Test case generation method and device and vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019100724A1 (en) | Method and device for training multi-label classification model | |
| CN109840531A (en) | The method and apparatus of training multi-tag disaggregated model | |
| CN111444952A (en) | Method and device for generating sample identification model, computer equipment and storage medium | |
| CN111444951B (en) | Sample recognition model generation method, device, computer equipment and storage medium | |
| KR20160041856A (en) | Systems and methods for performing bayesian optimization | |
| CN110046706B (en) | Model generation method and device and server | |
| CN111476268A (en) | Method, device, equipment and medium for training reproduction recognition model and image recognition | |
| CN111428557A (en) | Method and device for automatically checking handwritten signature based on neural network model | |
| CN112257808B (en) | Integrated collaborative training method and device for zero sample classification and terminal equipment | |
| CN111079780A (en) | Training method of space map convolution network, electronic device and storage medium | |
| CN109492093A (en) | File classification method and electronic device based on gauss hybrid models and EM algorithm | |
| CN115080749B (en) | Weak supervision text classification method, system and device based on self-supervision training | |
| CN111310743B (en) | Face recognition method and device, electronic equipment and readable storage medium | |
| CN113902944A (en) | Model training and scene recognition method, device, equipment and medium | |
| CN113822374B (en) | Model training method, system, terminal and storage medium based on semi-supervised learning | |
| CN115081613A (en) | Method and device for generating deep learning model, electronic equipment and storage medium | |
| CN117434429B (en) | Chip stability testing method and related device | |
| CN114116456A (en) | Test case generation method, system and computer readable storage medium | |
| CN113344079B (en) | Image tag semi-automatic labeling method, system, terminal and medium | |
| CN114781532A (en) | Evaluation method and device of machine learning model, computer equipment and medium | |
| CN111783688B (en) | Remote sensing image scene classification method based on convolutional neural network | |
| CN115641201A (en) | Data anomaly detection method, system, terminal device and storage medium | |
| CN114970732A (en) | Posterior calibration method and device for classification model, computer equipment and medium | |
| CN112559589A (en) | Remote surveying and mapping data processing method and system | |
| CN111026661A (en) | Method and system for comprehensively testing usability of software |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |