CN103823747B - The method of automatic regression test - Google Patents
The method of automatic regression test Download PDFInfo
- Publication number
- CN103823747B CN103823747B CN201210462280.9A CN201210462280A CN103823747B CN 103823747 B CN103823747 B CN 103823747B CN 201210462280 A CN201210462280 A CN 201210462280A CN 103823747 B CN103823747 B CN 103823747B
- Authority
- CN
- China
- Prior art keywords
- test
- regression test
- regression
- level
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of method of automatic regression test, start and run Step 1: returning, according to project concrete condition, regression test to variety classes test vector carries out classifying type classification no matter reason, for the different phase of layered verification, selection carries out module level, subsystem irrespective of size or system-level regression test respectively, and produces general information file and error message file；Post-processed Step 2: returning information, the regression test result for every kind of rank carries out statistical analysis, project is produced respectively and returns homepage, including project information and recurrence version, coverage rate；Generation module or return classification paging, including module classification list of types and by or failure test case collect；Produce the detailed regression result paging of each module, including each test case title, simulation run time, random number of times, if pass through, Fail Type statistics, simulation result general information file index.The present invention can improve the completeness of design verification flow path efficiency and checking.
Description
Technical field
The present invention relates to a kind of method that automatic regression test is carried out in the checking work of chip Front-end Design.
Background technology
Regression test occupies very important status in chip Front End Authentication.Because with chip complexity increasingly
It is high, it is necessary to head end test vector also constantly refine and increase, it is effective to its if none of effective regression testing method
Management, then whole checking work is possible to that a kind of poorly efficient working condition can be absorbed in.
Continuing to develop and complicating with verification technique and method, is surveyed for the recurrence for occupying important function in the verification
Examination also constantly proposes new demand and challenge, i.e., how within the limited time, ensures the quality of checking work, raising as far as possible
Front-end Design verifies flow scientific and efficientization, and guarantee is provided for the flow that chip is finally guaranteed the quality on time.
Traditional regression testing method be typically by institute operation in need test vector sequentially arrange, equivalent to criticize at
The each test vector of isolated operation is managed, the long period is separated by every time, or makes after larger renewal regression test of operation, and
Carry out inspection result finally by artificial or simple order.
Such regression testing method is relatively low in checking complexity, and can be met in the case of test vector negligible amounts will
Ask；But as design and verification environment become to become increasingly complex, the emphasis of test vector, the situation that quantity is all improved constantly
Under, traditional regression test is increasingly difficult to meet different designs Qualify Phase to regression test requirement.Such as test to
Measure increasing for quantity so that it is difficult to complete the regression test of all test vectors within an evening or short period, so ask
Topic will hide some dangers for when can not be fed back as early as possible for further design modification.Once returned in addition, being separated by long period operation
Test, can also prevent problem from feeding back as early as possible, design is in the function of constantly changing and already pass through（Performance）State,
And new modification is the problem of always be possible to bring new, vicious circle is so absorbed in, design work is ultimately resulted in and is absorbed in poorly efficient
Circulation.Further, if every time by artificial or simple command inspection result, substantially result can only be checked, it is impossible to provide each
The key message of test vector, except non-sequential checks the simulation result of each test vector, and if the test vector ratio of failure
In the case of more, such way also is difficult to implement；And it is also possible to produce artificial error detection by hand inspection result
As a result.
In order to make up the deficiency of conventional regression test, the challenge that the project that meets is proposed to checking work is, it is necessary to regression test
Can accomplish in the critical stage of design verification daily to latest update carry out regression test, it would be possible to defect or Failure elimination
In bud, while regression test can provide the key message of failure testing vector, meet the need for accelerating debugging, with this
To improve the efficiency of design verification work.
The content of the invention
The problem to be solved in the present invention is to provide a kind of method of automatic regression test, can improve design verification flow effect
Rate and the completeness of checking.
In order to solve the above technical problems, the method for the automatic regression test of the present invention, comprises the following steps：
Start and run Step 1: returning, according to project concrete condition, the regression test to variety classes test vector is entered
Reason is don't bother about in the classification of row classifying type, and for project reality, in the different phase of layered verification, selection respectively carries out module
Level, subsystem irrespective of size or system-level regression test, and produce general information file and error message file；
Step 2: returning information post processing, the regression test result for every kind of rank carries out statistical analysis, produces respectively
Project returns homepage, including project information and recurrence version, coverage rate；Generation module classification paging returns classification paging, bag
Include module classification list of types and by or failure test case collect；The detailed regression result paging of each module is produced,
Including each test case title, simulation run time, random number of times, if pass through, Fail Type statistics, simulation result is conventional
Message file is indexed.
The present invention is run using full-automatic script, by configuring the parameter of regression test, starts regression test；Verify engineering
Teacher is simply changed by starting or returning the configuration file of information post processing to returning, it is possible to be relatively easy to not
It is multiplexed between project；Variety classes test vector can be carried out to classification classification and don't bother about reason, and for project reality in layer
The different phase of secondary chemical examination card uses the regression test of different stage, the regression test of reasonable arrangement different phase.Surveyed returning
After examination terminates, automatic business processing regression test result can produce the recurrence report of user interface well, recurrence report bag
Include historical record and return every time, the key message of each module and each test vector facilitates engineer quickly to position defect
Or debugged, the checking flow path efficiency of chip Front-end Design and the completeness of checking can be effectively improved；Can be with checking ring
Border cooperates, and is engaged with third party's instrument and the design verification automated environment for subsidizing exploitation, easily with third party's work
Have an interface, the part as whole design verification environment forms unified design verification automation platform, automaticity compared with
High completion test result is checked.
Brief description of the drawings
The present invention is further detailed explanation with embodiment below in conjunction with the accompanying drawings：
Accompanying drawing is the method control flow chart of the automatic regression test.
Embodiment
The method of the automatic regression test is to be based on script and HTML（HTML）Developed etc. method, institute
The object of application is the checking flow progress automatic returning management for chip Front-end Design.In verification method and test and excitation not
In the case of disconnected complication, all test vectors can be more effectively managed, the checking flow of chip Front-end Design is improved, efficiently holds
Row design verification works.
Because automatized script performs the demand for durability, it is necessary to be joined as far as possible to project and verification environment
Numberization processing；The method of the automatic regression test is not independently present in verification environment, but needs to carry out some necessary
Basic handling.In short, mainly including two parts：It is the parameter configuration that project is normalized before project starts first,
Relevant design verification environment variable is set and configuration is called, so that future implements the automatic regression test by parameter extraction
Information needed for method；Secondly need to carry out rule settings, including bibliographic structure, naming rule in terms of verification platform and excitation
Deng.Automatic regression test can be achieved under the premise of herein.
Referring to shown in accompanying drawing, the method for the automatic regression test comprises the following steps：Return and start and run, return
Information is post-processed.
Described return starts and operating procedure, the parameter configuration that project is normalized before project starts, with installing
Verification environment relevant parameter and variable are counted, sets relevant design verification environment to call configuration；Pass through single command row or periodicity
Perform order（CRONTAB）Start automatic regression test；Call Version command synchronization design verification environment and design
Source code, and record relevant item recurrence version number；According to project concrete condition, the regression test to test vector is classified
Type is managed, according to order line is started, and carries out regression test type analysis, and select entrance according to regression test type analysis result
Corresponding regression test type；According to the regression test type for starting order line and entrance, regression test grade analysis, and root are carried out
Enter appropriate level regression test according to the selection of regression test grade analysis result.
The regression test type, including basic recurrence（SANITY）, it is quick to return（QUICK）, daily recurrence
（NIGHTLY）, it is overall to return（ROTATE）, single excitation recurrence（TEST）Five type regression tests.
Wherein：
The SANITY types regression test, is mainly used in quickly confirming after Method at Register Transfer Level code makes larger modification
The regression test of additional defect whether can be produced, i.e., after the Method at Register Transfer Level code update of design engineer, is confirmed whether
Other parts can be designed with the regression test for producing influence.
The QUICK types regression test, is generally used in design engineer every time to Method at Register Transfer Level code update
Quick regression test afterwards, it is similar with the SANITY types regression test function, but regression test scale is smaller.
The NIGHTLY types regression test, is the most frequently used a kind of regression test in project verification process, is generally used for
Carry out regression test every night checking middle and later periods of chip Front-end Design, produce daily regression test report, tracking and testing enters
Journey.
The ROTATE types regression test, for needing the time-consuming longer of operation confirmation before chip flow（That is chip
The recurrence of all excitations is carried out once before flow）, operation or need the last test for confirming back and forth to run without necessity to
Amount, before chip flow or vacation long period carry out ROTATE types regression test several times.
The TEST types regression test is used for supporting each test vector to carry out single artificial debugging.
Every kind of regression test type includes module level, subsystem irrespective of size, the regression test of system-level three ranks.
For the module level regression test and the regression test of subsystem irrespective of size, it is necessary to carry out test vector configuration successively, with
Machine number is configured, contoured configuration, the parameter such as coverage rate configuration.
Then need to carry out subclass analysis judgement for the system-level regression test, then select corresponding subclass to return
Test.
System-level regression test includes Method at Register Transfer Level（RTL）Regression test, field programmable gate array level（FL）Return
Return test and netlist level regression test（GSIM）Three subclass.Method at Register Transfer Level includes：Module level (BL), subsystem irrespective of size
(SS) it is, system-level（SL）And software-hardware synergism level（IC）Four sub- rank regression tests；The Method at Register Transfer Level is returned and surveyed
Examination is carried out after sub- rank selection, into corresponding sub- rank regression test.If netlist level, then it is when belonging to that to need analysis
The regression test of sequence species.Netlist level regression test includes：The regression test emulated in advance before placement-and-routing（PRE）, the worst feelings
Condition Regression is tested（WC）, the test of best-case Regression（BC）, regression test in typical case（TC）And it is low under best-case
Warm regression test（BCLT）, the sequential species of the standard cell lib that its classification situation should be provided according to wafer production factory increasing or
Reduce.Enter line interface configuration, encryption configuration, contoured configuration, coverage rate configuration to system-level regression test on this basis.According to
All configurations carried out before, run regression test, and produce general information file and error message file.
The excitation file designation form to emulation is needed to standardize before startup is returned, referring specifically to following table institute
Show.
Wherein, item domains can be directly obtained from design verification environmental variance；Level domain is needed according to different recurrence
Test grades are set according to pre-specified naming rule；Module domain needs according to the naming rule of module and verified
Function set；Functional domain needs simply to be set according to the characteristic point to be verified, without prespecified rule.
Before startup and operation is returned, need to standardize to the type information of emulation in verification platform, mainly
Including：
Module start information, belongs to some specific module for recording specific incentives file.
Module ending message, for record a module testing whether normal termination.
Start information is encouraged, the beginning of an excitation is used as.
Excitation-off information, for record one excitation normal termination, by or failure information, record one excitation
Whether pass through.
Simulation run temporal information, takes for recording a specific incentives.
If module level and subsystem level verification, using random number as excitation, in addition it is also necessary to number of repetition conduct
Random number information etc..Here is an example as explanation：
dv:Suite SCI7816 STARTED representation modules, which are returned, to be started；
dv:Test SCI7816_SEND STARTED represent that excitation emulation starts；
The average information of emulation is started；
The average information of emulation is started；
The average information of emulation is started；
dv:Test SCI7816_SEND PASSED represent that emulation passes through；
CPU time 114.70 represent the simulation run time；
dv:Test SCI7816_SEND repeat mumber=10 represent random number of times；
dv:Test SCI7816_SEND FINISHED represent that excitation emulation terminates；
dv:Suite SCI7816 FINISHED representation modules, which are returned, to be terminated.
The above-mentioned necessary standardization processing mainly carried out to naming rule and type information is, it is necessary to before recurrence starts
Complete, when carrying out regression test, then some module is directly customized according to input parameter selection and returned, some type is returned, or entirely
Chip is returned, following citing：
If carrying out SCI7816 module level recurrence, only need to run order line：Regress.cshSCI7816 is
Can.Wave file is produced without band other specification acquiescence, and random number of times is set for once.
If carrying out the full chip operation of subsystem irrespective of size, need to run order line：regress.cshsubsystem
Off1 wherein " off ", which represents to close, produces waveform option, and sets random number of times for once.
If carrying out SANITY type recurrence, it is only necessary to run order line：Regress.csh sanity, project
The startup script of regression test can quickly be returned automatically according to the typical excitation of option selection of inside configuration.
Returned it should be noted that merely just starting, such as to produce recurrence report, it is necessary to start the recurrence being subsequently noted
Test result handles script to produce report.
Described return starts and run, and using the startup script of project regression test, passes through single command row or periodicity
Perform order to be started, and can be universally used in module level, subsystem irrespective of size and system-level.The startup pin of the project regression test
This, supports automatic synchronization Version, and can judge it is module level regression test, subsystem irrespective of size time according to input parameter
Return test or system-level regression test；For every kind of rank regression test, support to produce general information file catalogue and mistake
Information file directory, collects general information and error message, and carry out different management to its different phase in recurrence respectively.Branch
Hold according to input parameter, choose whether that the source code of memory is encrypted.Support is determined according to coverage rate input parameter
It is no to need to produce coverage rate report.Support, according to Waveform switch parameter, whether configuration needs to produce waveform, and supports the help that malfunctions
Information etc..
The startup script of the project regression test, including project information, the checking of operation or recurrence rank, necessary road
Footpath parameter setting, instrument is called and version is set, and is cooperated with design verification automation platform.If new projects create or
When person needs the multiplexing to be transferred to a new projects, easy configuration can be carried out to the startup script of project regression test, hereafter only
Need running or enabling the startup script of the periodically execution order control project regression test daily, it is possible to realize back
Return test run and produce final regression test result.
Because all regression tests and interpretation of result process can be completed by simple command row, therefore can easily it make
Order is performed with periodicity to carry out automatic cycling operating, is coordinated different type regression test, can flexibly be arranged in pairs or groups, meet various
The need for regression test type.
The module level regression test starts script using the project regression test and started by configuration parameter, together
When the module that can test as needed it is different, option and installment module name realizes that the recurrence of some specific module is surveyed as input
Examination.It is main in double interface series products to support SCI7816, EEPROM controller（EECTRL）And radio frequency interface controller
（RFIF）Etc. module level verification and its regression test, need to configure the project regression test startup pin if having new module addition
This.According to the random parameter of input, the quantity of the packet of random test is configured, with the random parameter replacement test vector of input
In random number as random test packet；If the random parameter not inputted, automatically according in test vector with
Machine number produces the quantity of the packet of random test.According to the random parameter of input, the test vector of regression test is configured to,
If the random parameter not inputted, the test vector in all configuration files of acquiescence operation enters according to the test vector listed
Row regression test.
The subsystem irrespective of size regression test starts script using the project regression test and started by configuration parameter, can be with
The quantity of the packet of random test is set by inputting random parameter, if not inputting the value of random test, survey is called
The packet numerical value that examination vector is carried；The test vector for needing to return can be specified with input test vector lists file, if
Input test vector lists file is omitted, then calls the test vector list in configuration file automatically.
The system-level regression test starts script using the project regression test and started by configuration parameter, with module
Unlike level regression test and the regression test of subsystem irrespective of size, system-level regression test is directly emulated because of using firmware,
There is no the mechanism that the packet of random test is produced in test vector, it is not required that input random parameter.But system-level recurrence
Test is integrated with the regression test of the different subclass such as Method at Register Transfer Level, field programmable gate array level and netlist level.As returned
Test is returned to be used for Method at Register Transfer Level, then the project regression test, which starts script, need not configure netlist time sequential type parameter,
The design verification environment and script configuration parameter of the regression test of Method at Register Transfer Level system and software-hardware synergism level regression test are complete
Full multiplexing.As regression test is used for netlist level, then need to start the order line of script by sequential in the project regression test
Type is as elucidated before preferably, the worst as input parameter, typical case, low temperature regression test etc. under best-case.It is described to return
Start script be responsible for by the excitation of test in need behave, and after end of run according to input timing type not
Together, different routine information daily records and error message daily record are produced.Conventional and error message file designation rule is according to excitation
Name is named automatically plus the configuration of time sequential type, e.g., has run a SCI7816 system-level netlist test and excitation, reactionary slogan, anti-communist poster
Type is WC, then the journal file produced is entitled " TC.2901M4.SS.SCI7816_RECEIVE_sdfwc ", after facilitating
Continuous processing.
The regression test actuating section supported above for the present invention, when regression test operation is finished, it is necessary to regression test
As a result handled, that is, enter and return information post-processing step, specifically include following steps：
Step 1: being the regression test of that rank according to input parameter judgement.The present invention supports to include module level, subsystem
Irrespective of size, system-level regression test.
Step 2: collecting Version used in this regression test（Or other Versions）Source generation
Debugged or reappeared after code version number, convenience.
Step 3: according to current date, producing the file using current time as title, hereafter all processing and report will
Run in this file.
Step 4: operation regression test result treatment script enters to the general information file and error message file that are collected into
Row processing, and produce regression test detailed report.
Step 5: being decided whether to merge the coverage rate result of certain regression test according to input parameter, generation is covered
Lid rate report structure.
Step 6: according to the result that all regression tests are reported under all regression test catalogues, producing this project and returning survey
The top level page of report is tried, the top level page includes entry name, historical regression test report results link, and its coverage rate link
Deng.
The recurrence information post-processing stages only need to input regression test class information, and the class information can be system
One of time sequential type of level, subsystem or module level or netlist level, returns the entry program of information post processing
Current project information can be checked and recorded together with system information, regression test type parameter is passed into routine
Message file processing routine.
The general information file processing routine includes top layer general information file processing routine and bottom routine information text
Part processing routine.
The top layer general information file processing routine is used for producing project recurrence homepage（Including：Project information, returns version
This, coverage rate）, include returning analysis of version as shown in FIG. after the completion of operation is returned, return version and produce and produce covering
Rate link etc. can be completed in this stage, and it is to call third party's emulation tool automatically according to recurrence script startup to produce coverage rate
Parameter is produced, and the present invention simply directly invokes the coverage rate result produced.The top layer general information file processing routine
The module of required processing is classified according to the regression test type parameter of input, and module classification is at regression test result
Manage and set in script, therefore can be adjusted according to project difference.In top layer general information file processing routine header file certainly
Movable property gives birth to the report top layer HTML of project regression test（HTML）Formatted webpages file, this web page files are included
Project name and the recurrence version information produced according to run time and coverage rate link, and call at bottom general information file
Reason program is handled in detail.The old version of regression test can be collected simultaneously, time of different levels in each version is linked
Return test report, produce the total page of project regression test（Html format）.
The bottom general information file processing routine is used for producing the total page of regression test, including module classification paging or
Return classification paging（Including module classification list of types and by or failure test case collect）, and regression test will be produced
The detailed simulation result page of test vector of total page subordinate.Being shown in this stage needs first to carry out recursive type point
Analysis, returns grade analysis.Bottom general information file processing routine is according to the regression test type parameter and module information of input
Handled, generation module classification paging returns link, the type parameter one that wherein regression test type parameter and top layer are received
Cause, module information is generated by analysis by top layer general information file processing routine.Then each return is carried out to swash
Encourage the labor of result, analytic routines as depicted（log）Message file/mistake（err）Message file, including returned
Test grades are returned to analyze, analysis of simulation result, simulation run time series analysis, random number analysis；Produce returning in detail for each module
Sum up fruit paging, including each test case title, simulation run time, random number of times, if pass through, Fail Type is counted,
Simulation result general information file is indexed.With this produce each test vector comprising key message the detailed simulation result page and
According to the total page of the regression test of module classification；According to the simulation result of each test vector, vector report is produced, such as certain test
Vector emulation failure, then into error analysis branch, according at the beginning of general information file and error message file content carry out defect
Step is automatically positioned.
As a part for design verification automated environment, the automatic regression test method can be with other third party's works
Tool subsidizes the collaborative work of developing instrument script, and automatic regression test needs to support to include different emulators such as NCverilog
（Clang rises scientific and technological emulation tool）, VCS（Synopsys emulator）Deng；Different wave produces instrument such as VPD（Synopsys acquiescence production
Raw wave file）, FSDB（The scientific and technological wave file of source sheng, a reed pipe wind instrument, is purchased by Synopsys）, VCD（Most of instruments provide commercial city
The reference waveform file of support）Deng；The automatic regression test method supports Distributed Calculation instrument, such as SGE（Sun Company, Inc.
Distributed Calculation instrument）And LSF（The Distributed Calculation instrument that International Business Machines Corporation proposes）；When regression test is run,
Parallel execution instrument can be called（LSF or SGE）Distributed Calculation is carried out, regression test efficiency is improved, it is ensured that daily type is returned
Returning to complete；And energy and Synopsys（Synopsys）Emulate the coverage rate produced（coverage）It is combined, is each time
Version is returned to produce corresponding coverage rate situation report ".Ultimately produce html format form web page report can with it is cross-platform enter
Row is accessed.
Above by detailed description of the invention, the principle of the invention and application method are described, the present invention is not being departed from
In the case of principle, those skilled in the art can also make many deformations and improvement, such as be directed to different verification platforms, difference checking
Environment is, it is necessary to configure different parameters information, or the report produced can be comprising different key message etc., these optimize and changed
Change also should be regarded as protection scope of the present invention.
Claims (16)
1. a kind of method of automatic regression test, it is characterised in that comprise the following steps：Start and run, root Step 1: returning
According to project concrete condition, the regression test to variety classes test vector carries out classifying type classification no matter reason, actual for project
Situation selects to carry out module level, subsystem irrespective of size or system-level regression test respectively in the different phase of layered verification；Returning
After test run, general information file and error message file are produced；
Step 2: returning information post processing, the regression test result for every kind of rank carries out statistical analysis, and project is produced respectively
Return homepage, including project information and recurrence version, coverage rate；Generation module classification paging returns classification paging, including mould
Block sort list of types and by or failure test case collect；The detailed regression result paging of each module is produced, including
Each test case title, simulation run time, random number of times, if pass through, Fail Type statistics, simulation result routine information
File index；
Return to start and run to adopt described in step one and implement with the following method, the ginseng that project is normalized before project starts
Number configuration, configuration design verification environmental correclation parameter and variable, set relevant design verification environment to call configuration, pass through single command
Row periodically performs order startup automatic regression test；Call Version command synchronization design verification environment and set
Source code is counted, and records relevant item and returns version number；According to start order line, carry out regression test type analysis, and according to
The selection of regression test type analysis result enters corresponding regression test type；According to the regression test class for starting order line and entrance
Type, carries out regression test grade analysis, and enter appropriate level regression test according to the selection of regression test grade analysis result.
2. the method as described in claim 1, it is characterised in that regression test type described in step one, including following five kinds, point
It is not：
It is basic to return, after Method at Register Transfer Level code update, for being confirmed whether that influence can be produced on other parts design；
It is quick to return, after every time to Method at Register Transfer Level code update, produced for being confirmed whether that other parts can be designed
Influence, its regression test scale is less than the basic recursive type；
Daily recurrence, in the checking middle and later periods of chip Front-end Design, for carrying out regression test every night, produces daily return
Test report, tracking and testing process；
It is overall to return, the recurrence for carrying out once all excitations before chip flow, operation or need it is last confirm without
The test vector that back and forth runs of necessity, before chip flow or vacation carries out repeatedly overall recursive type regression test；
Single excitation is returned, for supporting each test vector to carry out single artificial debugging；
Every kind of regression test type includes module level, subsystem irrespective of size, system-level three ranks.
3. the method as described in claim 1, it is characterised in that：The module level regression test and subsystem irrespective of size regression test fortune
Before row, test vector configuration, random number configuration, contoured configuration and coverage rate configuration are carried out successively.
4. the method as described in claim 1, it is characterised in that：The system-level regression test includes：It is Method at Register Transfer Level, existing
Field programmable gate array level and netlist three subclass regression tests of level；Subclass selection is carried out to the system-level regression test
Afterwards, into corresponding subclass regression test；
The Method at Register Transfer Level regression test includes：Module level, subsystem irrespective of size, system-level and software-hardware synergism level are returned and surveyed
Examination；The Method at Register Transfer Level regression test is carried out after sub- rank selection, into corresponding sub- rank regression test；
The netlist level regression test, including：Regression test PRE, the worst case Regression emulated in advance before placement-and-routing is tested
Five kinds of low temperature regression test BCLT under WC, in typical case best-case Regression test b C, regression test TC and best-case
Time sequential type；Netlist level regression test is carried out after time sequential type selection, into corresponding time sequence type regression test；
Before the system-level regression test operation, line interface configuration, encryption configuration, contoured configuration and coverage rate configuration are entered successively.
5. the method as described in claim 1, it is characterised in that：To the excitation file designation form of emulation before startup is returned
Standardized；Wherein, item domains, are directly obtained from design verification environmental variance；Level domain, for different regression tests
Rank, is set according to pre-specified naming rule；Module domain, according to the naming rule of module and the function of being verified
Set；Functional domain, is set according to the characteristic point verified, without prespecified rule.
6. the method as described in claim 1, it is characterised in that：Before startup and operation is returned, to imitative in verification platform
Genuine type information is standardized；Including：
Module start information, belongs to some specific module for recording specific incentives file；
Module ending message, for record a module testing whether normal termination；
Start information is encouraged, the beginning of an excitation is used as；
Excitation-off information, for recording the normal termination of an excitation, by or failure information, whether an excitation lead to
Cross；
Simulation run temporal information, takes for recording a specific incentives；
If module level and subsystem irrespective of size regression test, using random number as excitation, then number of repetition is needed as random
Number information.
7. the method as described in claim 1, it is characterised in that：Returned described in step one and start and run, returned using project
The startup script of test, by single command row or periodically performs order and is started；The startup pin of the project regression test
This, supports automatic synchronization Version, judges it is module level regression test, the recurrence survey of subsystem irrespective of size according to input parameter
Examination or system-level regression test；For every kind of rank regression test, support to produce general information file catalogue and error message
File directory, collects general information and error message, and carry out different management to its different phase in recurrence respectively；Support root
According to input parameter, choose whether that the source code of memory is encrypted；Support decides whether need according to coverage rate input parameter
Produce coverage rate report；Support according to Waveform switch parameter, whether configuration needs to produce the type of waveform and waveform；Support out
Wrong help information.
8. method as claimed in claim 7, it is characterised in that：The startup script of project regression test, including project information, fortune
Capable checking or recurrence rank, necessary path parameter are set, and instrument is called and version is set, and platform is automated with design verification
Cooperate；
, can be to the startup pin of project regression test when creating if new projects or need the multiplexing to be transferred to a new projects
This progress easy configuration, hereafter only needs to running or enabling the periodically execution order control project regression test daily
Start script, it is possible to realize that regression test runs and produces final regression test result.
9. the method as described in claim 1,7 or 8, it is characterised in that：The module level regression test is returned using the project
Test starting script is returned to be started by configuration parameter, the module tested as needed is different, option and installment module name conduct
Input to realize the regression test of specific module；Need to configure the project regression test startup pin if having new module addition
This；
According to the random parameter of input, configure the quantity of the packet of random test, with the random parameter replacement test of input to
Random number in amount as random test packet；If the random parameter not inputted, automatically according in test vector
Random number produces the quantity of the packet of random test；
According to the random parameter of input, the test vector of regression test is configured to, if the random parameter not inputted, gives tacit consent to
The test vector in all configuration files is run, regression test is carried out according to the test vector listed.
10. the method as described in claim 1,7 or 8, it is characterised in that：The subsystem irrespective of size regression test is returned using project
Test starting script is started by configuration parameter, with the random parameter of input, sets the quantity of the packet of random test；If
Random parameter is not inputted, then the packet numerical value for calling test vector to carry；Being specified with input test vector lists file needs
The test vector of recurrence, if omitting input test vector lists file, calls the test vector in configuration file to arrange automatically
Table.
11. the method as described in claim 4,7 or 8, it is characterised in that：The system-level regression test is returned using the project
Test starting script is returned to start by configuration parameter；Wherein,
The design verification environment and script configuration parameter of Method at Register Transfer Level regression test and software-hardware synergism level regression test are complete
Full multiplexing；
Netlist level regression test, the then order line for starting script in the project regression test is joined time sequential type as input
Number, the project return start script be responsible for by the excitation of test in need behave, and according to defeated after end of run
Enter the difference of time sequential type, produce different routine information daily records and error message daily record；General information file and mistake are believed
Configuration of the breath file designation rule according to excitation name plus time sequential type is named automatically.
12. the method as described in claim 1, it is characterised in that：Support and third party's instrument or subsidy developing instrument script
Cooperate, support different emulators, different wave produces instrument, supports Distributed Calculation instrument, when regression test is run,
Selection and call as needed performs instrument and carries out Distributed Calculation parallel；And covering for generation can be emulated with Synopsys Synopsys
Lid rate is combined, and is that each recurrence version produces corresponding coverage rate situation report；In the top layer hypertext markup ultimately produced
The report of language html format form web page can be conducted interviews with cross-platform.
13. the method as described in claim 1, it is characterised in that：Information post processing is returned described in step 2, is specifically included following
Step：
Step 1, according to input parameter judge be that rank regression test；
Step 2, collect Version used in this regression test, it is convenient after debugged or reappeared；
Step 3, according to current date, produce the file using current time as title, hereafter all processing and report will herein
Run in file；
Step 4, operation regression test result treatment script are at the general information file and error message file that are collected into
Reason, and produce regression test detailed report；
Step 5, decided whether according to input parameter to merge the coverage rate result of certain regression test, produce coverage rate report
Accuse structure；
Step 6: according to the result that all regression tests are reported under all regression test catalogues, producing this project regression test report
The top level page of announcement, the top level page includes entry name, historical regression test report results link, the link of machine coverage rate.
14. the method as described in claim 1 or 13, it is characterised in that：Information post processing is returned described in step 2, it is only necessary to defeated
Enter regression test class information, the class information can be system-level, subsystem or module level or system-level middle net
One of time sequential type of table level；Returning the entry program of information post processing can be checked current project information and and be
System information is recorded together, and regression test level parameters are passed into general information file processing routine；
The general information file processing routine is included at top layer general information file processing routine and bottom general information file
Reason program.
15. method as claimed in claim 14, it is characterised in that：The top layer general information file processing routine produces project
Return homepage；The module of required processing is classified according to the regression test type parameter of input, and module classification is being returned
Set, be adjusted according to project difference in test result processing script；
The report top layer hypertext mark of project regression test is automatically generated in top layer general information file processing routine header file
Remember language html format web page files, the web page files include project name and the recurrence version information produced according to run time
Linked with coverage rate, and call bottom general information file processing routine to be handled in detail；Collect going through for regression test simultaneously
History version, links the regression test report of different levels in each version, produces the project of HTML html format
The total page of regression test.
16. method as claimed in claim 15, it is characterised in that：The bottom general information file processing routine produces recurrence
Test total page, including module classification paging or return classification paging, and will produce the test of the total page subordinate of regression test to
Measure the detailed simulation result page；
Handled according to the regression test type parameter and module information of input, generation module classification paging returns link, its
Middle regression test type parameter is consistent with the type parameter that top layer general information file processing routine is received, and module information is by pushing up
Layer general information file processing routine is produced by analysis；Each labor for returning and encouraging result is carried out, including is returned
Test grades are returned to analyze, analysis of simulation result, simulation run time series analysis, random number analysis；Produce returning in detail for each module
Sum up fruit paging；Produce the detailed simulation result page of each test vector comprising key message and the recurrence according to module classification
Test total page；According to the simulation result of each test vector, vector report is produced, such as certain test vector emulates failure, then entered
Enter error analysis branch, carrying out defect according to general information file and error message file content is tentatively automatically positioned.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210462280.9A CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210462280.9A CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103823747A CN103823747A (en) | 2014-05-28 |
| CN103823747B true CN103823747B (en) | 2017-09-15 |
Family
ID=50758828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210462280.9A Expired - Fee Related CN103823747B (en) | 2012-11-16 | 2012-11-16 | The method of automatic regression test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103823747B (en) |
Families Citing this family (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105487966B (en) * | 2014-09-17 | 2023-07-25 | 深圳市雅阅科技有限公司 | Program testing method, device and system |
| CN104679652B (en) * | 2015-02-15 | 2017-12-22 | 北京宇航***工程研究所 | A kind of analogue system method for generating test case based on Markov methods |
| CN104794053B (en) * | 2015-04-22 | 2017-12-01 | 烽火通信科技股份有限公司 | A kind of logical simulation checking regression test system and method based on random use-case |
| CN106291312A (en) * | 2015-06-12 | 2017-01-04 | 超威半导体(上海)有限公司 | A kind of method emulating for speed-up chip and debugging and chip test system |
| CN105158681A (en) * | 2015-08-07 | 2015-12-16 | 广州中大微电子有限公司 | Radio frequency identification reader chip verification method and system |
| CN105389200A (en) * | 2015-12-16 | 2016-03-09 | 浪潮(北京)电子信息产业有限公司 | Multi-group stimulation batch simulation method and system applied to super-large scale chip verification |
| CN105677996B (en) * | 2016-01-13 | 2018-10-23 | 浪潮集团有限公司 | A kind of method that regression test version exception source is determined in chip checking |
| CN105975924A (en) * | 2016-04-29 | 2016-09-28 | 杭州当虹科技有限公司 | Regression testing method for precisely recognizing ad content based on video frame statistics |
| CN106897485A (en) * | 2017-01-09 | 2017-06-27 | 北京中电华大电子设计有限责任公司 | Intelligence returns verification management system |
| CN107885925B (en) * | 2017-11-03 | 2021-06-29 | 中国电子科技集团公司第五十四研究所 | Register testing method in EDA (electronic design automation) verification stage |
| CN109240872B (en) * | 2018-08-08 | 2021-12-10 | 瑞芯微电子股份有限公司 | Statistical analysis method for coverage rate of chip verification key signals and storage medium |
| CN110232017A (en) * | 2019-04-30 | 2019-09-13 | 北京云迹科技有限公司 | Robot regression testing method and system under simulated environment |
| CN110348217A (en) * | 2019-05-28 | 2019-10-18 | 深圳壹账通智能科技有限公司 | Interface test method, device, electronic equipment and storage medium |
| CN111428431B (en) * | 2020-02-28 | 2024-02-02 | 中科亿海微电子科技（苏州）有限公司 | Automatic test and recording method and system supporting EDA software |
| CN111427794B (en) * | 2020-04-03 | 2023-05-16 | 飞腾信息技术有限公司 | Method, system and medium for accelerating simulation of memory component netlist |
| CN111859830A (en) * | 2020-06-30 | 2020-10-30 | 山东云海国创云计算装备产业创新中心有限公司 | Verification plan and report generation method, device, equipment and storage medium |
| CN112131827B (en) * | 2020-09-11 | 2023-03-28 | 山东云海国创云计算装备产业创新中心有限公司 | Chip testing method, system, equipment and storage medium |
| CN112015665B (en) * | 2020-09-30 | 2023-04-11 | 北京百度网讯科技有限公司 | Test verification backtracking method and device, electronic equipment and storage medium |
| CN112380127B (en) * | 2020-11-16 | 2022-11-22 | 海光信息技术股份有限公司 | Test case regression method, device, equipment and storage medium |
| CN112131512B (en) * | 2020-11-20 | 2021-02-09 | 中国人民解放军国防科技大学 | Method and system for website management script safety certification |
| CN112464596B (en) * | 2020-11-27 | 2022-11-15 | 海光信息技术股份有限公司 | Regression testing method, system, equipment and readable storage medium |
| CN112613255B (en) * | 2020-12-10 | 2023-01-24 | 海光信息技术股份有限公司 | Multi-environment collaborative regression method, device, equipment and storage medium |
| CN113704126A (en) * | 2021-09-02 | 2021-11-26 | 西安紫光国芯半导体有限公司 | Verification method and device, computer storage medium and processor |
| CN116306388B (en) * | 2023-05-23 | 2023-07-28 | 苇创微电子(上海)有限公司 | Automatic UVM verification platform free of path connection and construction method thereof |
| CN116776793B (en) * | 2023-08-22 | 2023-11-03 | 成都翌创微电子有限公司 | Multi-period path constraint verification method combining static time sequence analysis and pre-simulation |
| CN117034821B (en) * | 2023-10-09 | 2023-12-19 | 芯耀辉科技有限公司 | Regression verification method and medium for chip design front-end simulation verification |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178687A (en) * | 2007-11-26 | 2008-05-14 | 中兴通讯股份有限公司 | Software regression testing method |
| CN101329652A (en) * | 2008-07-30 | 2008-12-24 | 中兴通讯股份有限公司 | Regression test automatic system and method |
| CN102095448A (en) * | 2010-12-08 | 2011-06-15 | 江苏天泽信息产业股份有限公司 | Automatic testing system of production line of vehicle information terminal and testing method of system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7216343B2 (en) * | 2002-09-20 | 2007-05-08 | International Business Machines Corporation | Method and apparatus for automatic updating and testing of software |
-
2012
- 2012-11-16 CN CN201210462280.9A patent/CN103823747B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101178687A (en) * | 2007-11-26 | 2008-05-14 | 中兴通讯股份有限公司 | Software regression testing method |
| CN101329652A (en) * | 2008-07-30 | 2008-12-24 | 中兴通讯股份有限公司 | Regression test automatic system and method |
| CN102095448A (en) * | 2010-12-08 | 2011-06-15 | 江苏天泽信息产业股份有限公司 | Automatic testing system of production line of vehicle information terminal and testing method of system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103823747A (en) | 2014-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103823747B (en) | The method of automatic regression test | |
| CN110321281A (en) | Web test platform and test method based on mixing automated test frame | |
| CN104794057B (en) | A kind of crossing event automated testing method and device | |
| CN102331967B (en) | Method for managing chip verification test cases | |
| US6725435B2 (en) | Method and program product for completing a circuit design having embedded test structures | |
| CN102141962A (en) | Safety distributed test framework system and test method thereof | |
| CN102156784A (en) | Verifying environment patterned chip verifying method and device | |
| CN107463362A (en) | The method and system of lasting deployment based on multiple Jenkins | |
| CN105095059B (en) | A kind of method and apparatus of automatic test | |
| CN109684847A (en) | Self-repairing method, device, equipment and the storage medium of script loophole | |
| CN106841974B (en) | A kind of FPGA test platform and method | |
| CN105760299A (en) | Full-automatic script test method based on intelligent substation | |
| CN103246770A (en) | Activity graph model based system behavior simulation method | |
| CN105760296A (en) | Automation testing control method, device and terminal | |
| CN106648556A (en) | Front end and back end integrated development testing method and device | |
| CN106227654A (en) | A kind of test platform | |
| CN109491922A (en) | One kind being based on model-driven test method and apparatus | |
| CN101888318A (en) | System and method for testing performance index of communication system | |
| CN106371868A (en) | Automatic test method, system and test device | |
| CN116483730A (en) | Service system automatic test method based on domestic software and hardware and open source test tool | |
| CN109976990A (en) | It is a kind of for confirming the method and system of software test case priority | |
| CN107703773A (en) | A kind of method for testing software and device based on hardware-in-loop simulation system | |
| CN101765129A (en) | Network access test system and method for data value-added service | |
| CN110287092A (en) | A kind of electricity transaction system and its automatic test approach based on graphical interfaces | |
| CN104216703A (en) | Development method of embedded software system program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170915 Termination date: 20191116 |