CN114139475A

CN114139475A - Chip verification method, system, device and storage medium

Info

Publication number: CN114139475A
Application number: CN202111486430.5A
Authority: CN
Inventors: 谭黎敏; 李明慧; 宋捷
Original assignee: Shanghai Westwell Information Technology Co Ltd
Current assignee: Shanghai Westwell Information Technology Co Ltd
Priority date: 2021-12-07
Filing date: 2021-12-07
Publication date: 2022-03-04

Abstract

The invention provides a chip verification method, a system, equipment and a storage medium, wherein the method comprises the following steps: generating excitation data according to the model configuration parameters; establishing a software reference model and an imitation hardware reference model; respectively inputting the excitation data into the software reference model and the simulated hardware reference model, and acquiring software reference output and simulated hardware reference output; responsive to a comparison of the software reference output and the emulated hardware reference output, determining whether the emulated hardware reference model evaluates through a reference model; executing simulation verification according to the excitation data to obtain simulation output; and responding to the simulated hardware reference model, evaluating through a reference model, and comparing the simulated hardware reference output with the simulation output to obtain a chip verification result. The method is applied to simulation and verification of the neural network model inference chip, and the verification efficiency is improved.

Description

Chip verification method, system, device and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a method, a system, a device, and a storage medium for verifying a chip.

Background

Chip verification is an important ring in chip development, and design defects and errors can be found in time through the chip verification, so that the design of a chip is ensured to meet the expected functions. Chip verification always occupies a great labor and time cost due to the complexity and the cumbersome work of the system. In recent years, with the floor advancement of Artificial Intelligence algorithms, chip products specially performing Artificial Intelligence (AI) model reasoning acceleration have appeared. In order to enable the chip product to be suitable for complex and various models and application scenes and improve the market competitiveness of the product, the chip is designed more flexibly and has higher iteration speed, so that a light-weight and high-expansion customized chip verification system is vital to shortening the research and development period of the chip.

The mainstream framework of chip Verification is Universal Verification Methodology (UVM), which is a Universal Verification Methodology, and in such a conventional Verification structure, all components in the Verification structure need to be modified to participate even if the function of only one processor is verified. This makes the authentication resources expensive and takes relatively long time. The AI inference algorithm has more changeful and diversified characteristics according to different scenes, and in order to cope with the characteristics, the AI chip needs to have a more flexible verification platform to adapt to product research and development and iteration, and the complex modeling basis of a UVM library and an abstract level verification language based on System Verilog (SV) cannot well match the verification requirements of the deeply customized AI chip.

A chip verification method and system and a storage medium, CN111859831A, aiming at the artificial intelligence Core module (AI Core) of the convolutional neural network, the characteristics, weight, offset and other data characteristics need to be actively and continuously carried from an out-of-Core storage unit, and the chip verification system realized based on UVM verification methodology is adopted, so that the verification system has high reusability and configurability. However, the verification system depends on a plurality of UVM self components, verification personnel need to know the chip architecture and related protocols, and the interface building of each component is complex.

A method, a system and a verification platform for verifying a system-on-chip are disclosed, wherein CN113051855A observes the complex verification platform establishment through a UVM library and the high-consumption resource verification condition, and provides an improved method for verifying the system-on-chip by establishing various component libraries including an interface protocol component and a bus protocol component, software libraries for running each processor of the system-on-chip and an excitation library of corresponding components. Although the method relieves the dependence of a verification platform based on a UVM library, a plurality of lower-layer chip design concepts irrelevant to chip verification are introduced, the whole verification process is complex, and the staged verification requirement of chip development is not facilitated.

CN113220518A discloses a chip verification system and a chip verification method, which package the chip verification process into several independent modules, such as an excitation module and a configuration module, according to the special format of the chip to be verified, to ensure the accuracy of chip function verification. The verification method only aims at the functional verification requirement of the chip, and the function is relatively single.

The AI algorithm model of the industry at present has the characteristics of deep customization: for different application scenes and data, algorithm engineers can design different model architectures, the models can be composed of different operators, such as a convolutional neural network based on computer vision, and can be divided into convolutional operators, pooling operators and the like, chip developers can initially develop chips by taking the operators as units, develop accelerator cards capable of covering simple inference models, and then continuously perform product iteration with development of business and derivation of AI models to have more supported operator libraries, so that more inference models are supported. Chip verification is an important component of chip development, and determines the development cycle of the chip. Aiming at the research and development characteristics of the AI chip, the currently common chip verification method or platform has more chip bottom layer concepts, and is not beneficial to the rapid start of verification personnel; the verification link is complex, one small change needs global adjustment, the consumption of verification resources is high, and the later function expansion and code reuse are limited.

In addition, the AI chip is diverse and diverse, model performance is often optimized by ingenious design at an algorithm level, some adjustments are made if hardware design cannot be completely supported in implementation, and the adjusted scheme needs to backtrack and evaluate the influence degree on the overall model performance, so that the process of developing the AI chip product from scheme design to chip verification is continuously iterative and interactive. Most of the System Verilog (SV) -based compiling platforms for abstract level verification languages and C are designed based on object-oriented programming characteristics and verification platforms, so that the correlation among all links is ignored, and the workload of chip research and development is increased to a certain extent.

Therefore, how to optimize the chip verification to reduce the complexity of the chip verification, improve the flexibility and expansibility of the chip verification, and reduce the redundancy of the chip verification is a technical problem to be solved urgently by those skilled in the art,

disclosure of Invention

Aiming at the problems in the prior art, the invention aims to provide a chip verification method, a system, equipment and a storage medium, which optimize chip verification so as to reduce the complexity of chip verification, improve the flexibility and expansibility of chip verification and reduce the redundancy of chip verification.

The embodiment of the invention provides a chip verification method, which comprises the following steps:

generating excitation data according to the model configuration parameters;

establishing a software reference model and an imitation hardware reference model;

respectively inputting the excitation data into the software reference model and the simulated hardware reference model, acquiring a software reference output and a simulated hardware reference output, and determining whether the simulated hardware reference model passes the reference model evaluation or not based on the comparison of the software reference output and the simulated hardware reference output;

executing simulation verification according to the excitation data to obtain simulation output;

and responding to the simulated hardware reference model, evaluating through a reference model, and comparing the simulated hardware reference output with the simulation output to obtain a chip verification result.

In some embodiments of the present application, the generating the excitation data according to the model configuration parameters comprises:

generating first excitation data and second excitation data, wherein the first excitation data is generated based on a trained model and the second excitation data is generated based on a stochastic function,

the software reference model, the simulated hardware reference model and the simulation verification all use the second excitation data, and the first excitation data is used for verifying the software development performance of the chip.

In some embodiments of the present application, the inputting the excitation data into the software reference model and the simulated hardware reference model, respectively, and the obtaining a software reference output and a simulated hardware reference output includes:

respectively inputting the excitation data into the software reference model and the simulated hardware reference model to obtain the software intermediate layer output of the software reference model and the simulated hardware intermediate layer output of the simulated hardware reference model;

acquiring the intermediate output difference between the software intermediate layer output and the simulated hardware intermediate layer output;

acquiring software performance parameters of the software reference model and simulated hardware performance parameters of the simulated hardware reference model;

acquiring performance differences of the software performance parameters and the simulated hardware performance parameters;

and judging whether the middle layer of the simulated hardware reference model passes the evaluation of the reference model or not according to the functional relation between the middle output difference and the performance difference.

In some embodiments of the present application, said performing simulation verification based on said excitation data, and obtaining a simulation output includes:

and executing register transmission level simulation verification according to the excitation data to obtain register transmission level simulation output.

and executing the prototype simulation verification of the field programmable gate array according to the excitation data to obtain the prototype simulation output of the field programmable gate array.

In some embodiments of the present application, in response to the chip verification result being a second verification evaluation failure, modifying the code of the simulation verification to perform a regression test;

and responding to the chip verification result that the second verification evaluation is passed, and saving the code of the simulation verification to perform regression testing.

In some embodiments of the present application, the generating the excitation data according to the model configuration parameters further comprises:

and carrying out parameter configuration of a model according to the excitation data, and carrying out data compilation according to the parameter configuration so as to execute software development of the chip.

According to another aspect of the present application, there is also provided a chip verification system, including:

the excitation generating module is used for generating excitation data according to the model configuration parameters;

the reference model module is used for establishing a software reference model and an imitation hardware reference model;

a reference model output module, configured to input the excitation data into the software reference model and the simulated hardware reference model, respectively, obtain a software reference output and a simulated hardware reference output, and determine whether the simulated hardware reference model passes a reference model evaluation based on a comparison between the software reference output and the simulated hardware reference output;

the simulation module is used for executing simulation verification according to the excitation data to obtain simulation output;

and the verification result module is used for responding to the simulated hardware reference model and evaluating through a reference model, and comparing the simulated hardware reference output with the simulation output to obtain a chip verification result.

An embodiment of the present invention further provides a chip verification apparatus, including:

a processor;

a memory having stored therein executable instructions of the processor;

wherein the processor is configured to perform the steps of the chip verification method via execution of the executable instructions.

The embodiment of the invention also provides a computer readable storage medium for storing a program, and the program realizes the steps of the chip verification method when being executed by a processor.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

The chip verification method, the system, the equipment and the storage medium have the following beneficial effects:

according to the method, a software reference model and an imitation hardware reference model are established, excitation data are respectively input into the software reference model and the imitation hardware reference model, and whether the imitation hardware reference model is evaluated through a reference model or not is determined based on comparison between the software reference output and the imitation hardware reference output. The method and the device can realize automatic evaluation of the simulated hardware reference model, when the simulated hardware reference model fails to be evaluated, the simulated hardware reference model can be directly adjusted, subsequent simulation verification is not required to be executed, and when the simulated hardware reference model passes the evaluation, the subsequent simulation verification can be executed based on the simulated hardware reference model, so that the chip verification is optimized, the complexity of the chip verification is reduced, the flexibility and the expansibility of the chip verification are improved, and meanwhile, the redundancy of the chip verification is reduced.

Drawings

Other features, objects and advantages of the present invention will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, with reference to the accompanying drawings.

FIG. 1 is a flow chart of a chip verification method according to an embodiment of the invention;

FIG. 2 is a flow chart of model evaluation according to an embodiment of the present invention;

FIG. 3 is a flow chart of a chip verification method according to an embodiment of the invention;

FIG. 4 is a schematic diagram of a chip development lifecycle of an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a chip verification system according to an embodiment of the present invention;

FIG. 6 is an interaction diagram of the modules of the chip verification system according to an embodiment of the invention;

FIG. 7 is a schematic structural diagram of a chip verification device according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.

As shown in fig. 1, an embodiment of the present invention provides a chip verification method, which is applied to simulation and verification of a neural network model inference chip, and the method includes the following steps:

step S110: excitation data is generated according to the model configuration parameters.

Specifically, the Artificial Intelligence (AI) algorithm is different from the conventional algorithm in that the form is varied and the version iteration speed is fast, and the AI algorithm is represented by a larger number of layers and a larger variety of types of algorithms. Different algorithms are combined to form different algorithm models. The operator is composed of a plurality of hyper-parameters, a large number of test cases can be generated by randomly combining the hyper-parameters, and the characteristics require that a chip verification framework of the AI algorithm is lighter and has high flexibility.

Thus, in the present application, the model configuration parameters may include the number of layers of the neural network, the type of each layer of the operator, the shape of the input data, the shape of the weighted data, the parameters of the operator, and the like. Specifically, the operator is an individual functional unit in the neural network model, and a convolutional neural network model is taken as an example, and includes a convolution operator, a pooling operator, a full join operator, and the like.

In some embodiments, the model configuration parameters may define operator parameters and test parameters of at least one test case, and the test parameters further include a test case number. The test case may be a functional test directed to an operator, or a test directed to a model, where one model includes a plurality of operators, and the operators in the model are connected according to a certain order. Different operator parameters may form different operator test cases. Taking a common convolution operator as an example, the operator parameters include the length, width, channel number, convolution kernel size, convolution step length, zero padding mode and the like of the output. When one test case is used for testing the operator, the operation mode of the test case is the single operator test, so that the function of the single operator can be tested, and the problem tracking and positioning are convenient when the problem is verified.

Step S120: and establishing a software reference model and an imitation hardware reference model.

Specifically, in the present application, the software reference model established in step S120 is implemented based on a pure software idea of an algorithm, and this part does not need to consider a lower-layer hardware logic, and each step directly calls a general algorithm library of the software, so as to concentrate on the algorithm idea itself, and output data as a real reference standard. The simulated hardware reference model established in step S120 is realized by software simulation based on a hardware concept. The part is faithful to the design idea of hardware, and the function of realizing the algorithm is realized according to the design idea of a chip bottom layer in each step. The output of the simulated hardware reference model is compared with the subsequent simulation output result to realize the verification of the chip.

In some specific implementations, intermediate position observation points (such as set intermediate layer/intermediate operator observation points) can be set in advance for the simulated hardware reference model, and when all sub-modules of the chip are combined and joint-tuned, the problem can be directly positioned conveniently.

Step S130: and respectively inputting the excitation data into the software reference model and the simulated hardware reference model, acquiring a software reference output and a simulated hardware reference output, and determining whether the simulated hardware reference model passes through the reference model evaluation or not based on the comparison of the software reference output and the simulated hardware reference output.

Specifically, the design of the AI operator is flexible and changeable, and is limited by the complexity of hardware logic, and sometimes complete software step implementation cannot be achieved, so that step S130 evaluates the influence of the overall model performance by comparing the pure software output of the software reference model and the simulated hardware output of the simulated hardware reference model, for example, how much influence is exerted on the performance of the entire model by a small change in the simulated hardware and pure software output of a certain middle layer, and determines whether the small change is within an acceptable range (set range) by using the evaluation standard, if so, the simulated hardware reference model constructed in step S120 is used as a reference standard for chip function verification, and a next function verification process is performed. If the error of the evaluation result is larger, the chip design idea has defects, and further adjustment is needed.

In particular, the criteria for model evaluation can be measured in two dimensions, one for true value deviation and one for model accuracy deviation. For example, the actual value deviation may be a percentage of an absolute value of a subtraction between a value of the software reference output and a value of the hardware reference output and an absolute value of the software reference output, and if the absolute value exceeds a set threshold (e.g., 10%), the error is considered to be large, and the hardware implementation needs to be adjusted. The model accuracy deviation can be regarded as a performance evaluation standard common to model performances of different tasks, for example, average intersection ratio (mIOU) is used for dividing a task model, mIOU of a software reference scheme and mIOU of a hardware reference scheme are respectively calculated according to model output, if the difference between the mIOU of the software reference scheme and the mIOU of the hardware reference scheme is larger than a set threshold (such as 0.02), an error is considered to be large, and a hardware implementation scheme needs to be adjusted.

Step S140: and executing simulation verification according to the excitation data to obtain simulation output.

Specifically, one or more emulation verification methods, such as register transfer level emulation verification, FPGA prototype verification, etc., may be implemented in step S140, which is not limited in this application.

Step S150: and responding to the simulated hardware reference model, evaluating through a reference model, and comparing the simulated hardware reference output with the simulation output to obtain a chip verification result.

Referring now to FIG. 2, FIG. 2 is a flow chart of model evaluation according to an embodiment of the present invention. Fig. 2 shows the following steps in total:

step S131: and respectively inputting the excitation data into the software reference model and the simulated hardware reference model to obtain the software intermediate layer output of the software reference model and the simulated hardware intermediate layer output of the simulated hardware reference model.

Step S132: and acquiring the intermediate output difference between the software intermediate layer output and the simulated hardware intermediate layer output.

Step S133: and acquiring the software performance parameters of the software reference model and the simulated hardware performance parameters of the simulated hardware reference model.

Step S134: and acquiring the performance difference between the software performance parameter and the simulated hardware performance parameter.

Step S135: and judging whether the middle layer of the simulated hardware reference model passes the evaluation of the reference model or not according to the functional relation between the middle output difference and the performance difference.

Specifically, in this embodiment, the software middle layer output and the emulated hardware middle layer output are the same layer of the AI model. The intermediate layer may be, for example, the aforementioned observation point. In some variations of the present application, the intermediate layer from step S131 to step S135 may be any intermediate operator. Thus, the present application may enable flexible, different types of reference model evaluation.

Specifically, in the above step, the intermediate output difference may be, for example, a difference between the software intermediate layer output and the hardware-simulated intermediate layer output. In some variations, the intermediate output difference may also be a difference value obtained by performing normalization processing based on a difference value between the software intermediate layer output and the hardware-simulated intermediate layer output.

Specifically, in the above steps, the performance parameter may be, for example, one or more of an accuracy, a recall rate, and an execution speed of the model, which is not limited in the present application.

Specifically, in the above step, an intermediate output difference of a plurality of intermediate layer performance differences may be obtained, so that a functional relationship between each intermediate output difference and the performance difference can be obtained through fitting, and thus, an intermediate layer (intermediate operator) having the largest influence on the performance difference is located, so that a designer can adjust the intermediate layer (intermediate operator) conveniently.

In further variants of the present application, the software reference model and the simulated hardware reference model may also be evaluated in their entirety. For example, when the difference value between the overall output of the software reference model and the overall output of the simulated hardware reference model is greater than a set threshold range, the simulated hardware reference model is considered to be not evaluated by reference; and when the difference value between the overall output of the software reference model and the overall output of the simulated hardware reference model falls into a set threshold range, the simulated hardware reference model can be considered to be evaluated through reference.

Specifically, the evaluation of the software reference model and the middle layer (intermediate operator) of the simulated hardware reference model, and the overall evaluation of the model can be set as required, so as to realize combined evaluation. For example, in some implementations, if the performance difference between the software performance parameter and the simulated hardware performance parameter is greater than a set threshold range, the simulated hardware reference model may be deemed to have failed the reference evaluation; when the performance difference between the software performance parameter and the simulated hardware performance parameter falls into a set threshold range, further judging whether the difference value between the overall output of the software reference model and the overall output of the simulated hardware reference model is larger than the set threshold range, and if so, determining that the simulated hardware reference model does not pass the reference evaluation; and when the difference value between the overall output of the software reference model and the overall output of the simulated hardware reference model falls into a set threshold range, the simulated hardware reference model can be considered to be evaluated through reference. For example, in other specific implementations, when the performance difference between the software performance parameter and the simulated hardware performance parameter is greater than the set threshold range, it may be determined that the simulated hardware reference model does not pass the reference evaluation, and meanwhile, the above steps S131 to S135 are performed to locate an intermediate layer (intermediate operator) having the largest influence on performance, so as to be referred by a chip setting person. The present application is not limited thereto, and the specific evaluation manner can be set as required.

Referring now to FIG. 3, FIG. 3 is a flow chart of a chip verification method according to an embodiment of the invention;

step S210: first excitation data and second excitation data are generated according to the model configuration parameters.

Specifically, the excitation data is generated by 2 interfaces, one is a trained real model data interface, and is used for providing first excitation data; the other is a random digital-to-analog interface for providing second excitation data. A random number interface, which uses a Numpy library (python-based mathematical function library) random function to generate different excitation data according to model parameter information in each simulation, so as to meet the boundary value and coverage rate test requirements in the simulation verification stage; and the real model data interface is the existing trained excitation data. The real model data interface can be used for verifying the overall performance of the network developed by corresponding software of the products at the middle and later stages in the step S280, and a more intuitive evaluation is realized through the expression of the real network and the calculation force table of other products.

Step S220: and establishing a software reference model and an imitation hardware reference model.

Specifically, the establishment of the software reference model and the simulated hardware reference model is performed based on the second excitation data. The software reference model established in step S220 is realized based on a pure software idea of an algorithm, the part does not need to consider a lower-layer hardware logic, a general algorithm library of the software is directly called in each step, the algorithm idea itself is more focused, and output data is used as a real reference standard. The simulated hardware reference model established in step S220 is realized by software simulation based on a hardware concept. The part is faithful to the design idea of hardware, and the function of realizing the algorithm is realized according to the design idea of a chip bottom layer in each step. The output of the simulated hardware reference model is compared with the subsequent simulation output result to realize the verification of the chip.

Step S231: and respectively inputting the excitation data into the software reference model and the simulated hardware reference model, acquiring a software reference output and a simulated hardware reference output, and determining whether the simulated hardware reference model passes through the reference model evaluation or not based on the comparison of the software reference output and the simulated hardware reference output.

Step S232: and judging whether the simulated hardware reference model passes the reference model evaluation.

If the determination in step S232 is no, the process returns to step S220. If the determination in step S232 is yes, step S253 is executed.

Specifically, while the software reference model and the simulated hardware reference model are established, step S240 may be executed synchronously: and (5) parameter configuration.

Specifically, for the design of the AI model reasoning acceleration chip, the configuration required for each task start comprises the following steps: register information, model weight data, input data. The register information refers to a data register, an index register, a pointer register and the like which are designed in advance and used for telling the chip task information; the model weight data is the data of the AI inference model, and each customized model has a different set of data. The input data are variables of the custom models, and each custom model is used for realizing a desired reasoning result for different input data. An AI accelerating chip can deploy a plurality of sets of models of different reasoning tasks through register information, and one set of reasoning models realizes reasoning results of different input data. Data configuration preparation of a set of models may be implemented at step S240.

Step S251: and executing register transmission level simulation verification according to the excitation data to obtain register transmission level simulation output.

Specifically, the excitation data used in step S251 is second excitation data.

Specifically, after the chip design is completed, a developer may implement logic design of a module by using a Hardware Description Language (HDL) according to a function and an interface requirement, so as to form a Register-transfer Level (RTL) code, and then, step S251 may perform function verification, which is also referred to as RTL simulation verification. Specifically, step S251 first implements the drive preparation for the functional simulation: and according to the simulated interface design, realizing the format conversion of the second excitation data, and storing the second excitation data in a text form. Then, a simulation program is started to perform register transmission level simulation verification based on the second excitation data which is converted by the format, and register transmission level simulation output is obtained.

Step S252: and executing the prototype simulation verification of the field programmable gate array according to the excitation data to obtain the prototype simulation output of the field programmable gate array.

Specifically, the excitation data used in step S252 is second excitation data.

Specifically, the Field-Programmable Gate Array (FPGA) prototype verification is to transplant an RTL code to a FPGA to verify the function of an Application Specific Integrated Circuit (ASIC), and after the basic function verification of a chip is passed, development of driving and Application can be started. The FPGA verification stage is a necessary stage after the functional logic verification is completed, the verification of the process includes a real clock delay condition, and the functional defects caused by the failure possibly caused by the delay problem in an actual gate-level circuit can be detected. In step S252, first, data conversion is implemented, which may implement format conversion of the second excitation data according to the interface design of the FPGA prototype verification, and the second excitation data is stored in a text form, and then, a chip workflow is started through the driver configuration, so that a simulation output is obtained through the chip workflow.

In some specific implementations, step S252 and step S251 may be executed synchronously or later than step S251, and further variations may be implemented in the present application, which is not described herein again.

Step S253: and comparing the simulated hardware reference output with the simulated output, and judging whether the outputs are consistent.

If the determination in step S253 is yes, step S261 of maintaining the emulation verified code (register transfer level code) is executed.

If the determination in step S253 is no, step S263 is executed: the emulation verified code (register transfer level code) is modified.

To ensure more efficient debugging and prevent inadvertent impact on other modules or code logic, regression testing may be performed on the cases that have been tested through step S262. Then a step of archiving the information by the data set is also necessary. Therefore, the classification of the test cases and the archiving and saving of the corresponding incentive data can be realized.

Specifically, for step S261 to step S263, if the test case fails to be verified, the defect code is located by the observation point agreed in advance, and then the RTL code is repaired. For a large-scale complex logic design, the involvement among modules is large, and when a developer repairs codes, regression testing is needed to prevent the situation that other passed test cases are affected in order to repair a bug. The method can adopt a control variable method to assist debugging and repairing of the code. And after the RTL code is modified, retesting the original test case by taking the archived data of the verification set as a reference, and only when the RTL code is consistent with the original data, considering that the RTL is successfully repaired and integrating the new code into the code engineering. Otherwise, the repair is continued, and no new test case flow is performed.

Step S270: and compiling data according to the parameter configuration.

Specifically, step S270 may correspond to a partial work of the later software development of step S280. After the chip is developed, it is necessary to implement a more general and automatic chip software application design, so step S270 may be a conversion and compilation tool of the inference model, and implement automatic conversion of model parameters and automatic configuration of register information. Step S270 may multiplex the functions of the parameter configuration module, add the allocation of the offset address, and package into a binary file containing the inference model and the chip information, which is used as a software call interface to complete the preprocessing step of software development.

Step S280: and performing later chip software development.

Referring now to fig. 4, fig. 4 is a schematic diagram illustrating a chip development lifecycle, according to an embodiment of the invention. Fig. 4 shows step S310: designing a scheme; step S321: verifying the scheme; step S322 functional simulation verification: step S323: verifying an FPGA prototype; step S324 compiling and developing; and step S330: and (5) software development.

In particular, the verification platform portal may accept two parameters, a verification pattern and a model parameter address. Different code flows are entered through setting of the verification mode, and the program running efficiency is improved. The agreed code flow is as follows:

step S321: scheme verification: and (4) evaluating the influence of the software pure algorithm implementation scheme and the hardware-thought-imitating implementation scheme on the model performance to assist the design of the chip scheme. In this mode, the program runs with the output of the scenario evaluation as the end point, and no other processes are performed.

Step S322: function verification mode: the excitation data can drive two branches through the construction of a reference model and the functional simulation, and then the comparison verification is carried out to verify the simulation result. If the output comparison is consistent, the test cases passing the verification are recorded and stored for being used as the verification set maintenance of the modification reference in the debugging stage. And if the output comparison is inconsistent, the RTL developer performs tracking repair on the implementation process. In this mode, the program runs with the comparison output as the end point. No other flow is performed.

Step S323: FPGA prototype verification mode: the excitation data enters a comparison verification module through the construction of a reference model and the driving of two branches of FPGA prototype verification, and enters a verification set for maintenance according to a verification result in the same steps as the steps in the later functional verification mode. In this mode, the program operation also takes the comparison output as the end point.

Step S324: compiler mode: in the mode, excitation data can directly enter the compiler module only through the branch of the parameter configuration module, and a binary format file containing model parameters and chip register information is output. In this mode, the program function is mainly to output the package file, and may not pass through other verification modules.

Specifically, in the application, the whole verification platform can be built in a pycharm development environment by using a python language. This ensures that each flow of verification is more intuitive and visible without going through the complex compilation environment of traditional compilation platforms and languages. Particularly, for the development of the AI reasoning acceleration chip, after a lifecycle is completed, steps S321 to S324 support parallel execution, and at this time, project managers arrange personnel to perform work at different stages according to project schedule requirements, and developers are clearer in task flow of themselves. Based on the set of verification platform, the development rhythm is more compact, and the dispatching efficiency of personnel is higher.

Therefore, in the application, the design period of the chip is shortened by referring to the scheme verification function of the model; sharing parameter configuration, multiplexing the configuration information function modules in different stages of verification, and combining with later software development requirements, improving code reuse rate and reducing development and maintenance cost; the idea of designing a verification platform by taking AI product rapid iteration as a guide is adopted, so that several important flows developed by a chip can be operated and configured in a parallelization manner, the personnel distribution efficiency is improved, the interaction requirements among the flows are met, and the development period is effectively shortened; based on a pycharm integrated platform and a python language, compiling time of a traditional method is eliminated, and chip verification and debugging are more visual and efficient.

Further, in the application, with the design idea oriented to the iterative process of the chip product, the developers for chip design, development, verification and compiling can share one set of platform, so that the cooperation of the front end and the rear end of the chip development is smoother, the work is more efficient, and the chip development cycle is favorably shortened; the verification platform is built based on the pychar integrated development environment by using the python language, so that the whole chip verification platform is built more simply and is lighter. The high cohesion and low coupling characteristics of software development are realized through the program main entry control parameters, and the consumption of resources in the verification process is reduced.

The foregoing merely illustrates a number of implementations of the present application, which can be implemented individually or in combination, and the present application is not limited thereto.

As shown in fig. 5, an embodiment of the present invention further provides a chip verification system, configured to implement the chip verification method, where the system 400 includes:

the excitation generating module 410 is configured to generate excitation data according to the model configuration parameters;

the reference model module 420 is used for establishing a software reference model and a simulated hardware reference model;

the reference model evaluation module 430 is configured to input the excitation data into the software reference model and the simulated hardware reference model respectively, obtain a software reference output and a simulated hardware reference output, and determine whether the simulated hardware reference model passes a reference model evaluation based on a comparison between the software reference output and the simulated hardware reference output;

the simulation module 440 is configured to perform simulation verification according to the excitation data to obtain a simulation output;

the verification result module 450 is configured to respond to the simulated hardware reference model and evaluate through a reference model, and compare the simulated hardware reference output with the simulation output to obtain a chip verification result.

The chip verification system can be deployed on a PC or a server to construct a chip verification environment, and the chip verification environment determines whether the simulated hardware reference model is evaluated through a reference model or not by establishing a software reference model and a simulated hardware reference model and respectively inputting excitation data into the software reference model and the simulated hardware reference model based on comparison between the software reference output and the simulated hardware reference output. The method and the device can realize automatic evaluation of the simulated hardware reference model, when the simulated hardware reference model fails to be evaluated, the simulated hardware reference model can be directly adjusted, subsequent simulation verification is not required to be executed, and when the simulated hardware reference model passes the evaluation, the subsequent simulation verification can be executed based on the simulated hardware reference model, so that the chip verification is optimized, the complexity of the chip verification is reduced, the flexibility and the expansibility of the chip verification are improved, and meanwhile, the redundancy of the chip verification is reduced.

The functions of the modules in the chip verification system of the present invention can be implemented by using the specific implementation of the above steps, and are not described herein again.

As shown in fig. 6, which is an interaction diagram of each module of the embodiment of the present application:

the excitation generation module 410 is configured to generate first excitation data and second excitation data according to the model configuration parameters.

The reference model module 420 is used to establish a software reference model and a simulated hardware reference model.

The reference model output module 430 is configured to input the excitation data into the software reference model and the simulated hardware reference model, respectively, obtain a software reference output and a simulated hardware reference output, and determine whether the simulated hardware reference model passes a reference model evaluation based on a comparison between the software reference output and the simulated hardware reference output.

Then, whether the simulated hardware reference model is evaluated through a reference model is judged. If the model is not evaluated by the reference model, the model is re-established by returning to the reference model module 420. And if the evaluation is passed through the reference model, comparing and verifying the next step.

In particular, while the reference model module 420 builds the software reference model and the emulated hardware reference model, the parameter configuration module 460 may perform parameter configuration synchronously.

And the functional simulation verification module 441 is configured to execute register transmission level simulation verification according to the excitation data, and obtain register transmission level simulation output.

The FPGA prototype verification module 442 is configured to perform field programmable gate array prototype simulation verification according to the excitation data, and obtain field programmable gate array prototype simulation output.

Then, whether the output of the simulated hardware reference model is consistent or not is compared according to the functional simulation verification module 441 and the functional simulation verification module 441. If so, the emulated verified code (register transfer level code) is maintained by the verification set maintenance module 491. If not, the emulation verified code (register transfer level code) is modified by emulation verification modification module 493. Regression testing 492 is then performed.

The compiler module 480 is configured to compile data according to the parameter configuration.

Fig. 5 and fig. 6 are only schematic diagrams respectively showing the chip verification system provided by the present invention, and the splitting, merging and adding of modules are within the protection scope of the present invention without departing from the concept of the present invention. The chip verification system provided by the present invention can be implemented by software, hardware, firmware, plug-in and any combination thereof, and the present invention is not limited thereto.

The embodiment of the invention also provides chip verification equipment, which comprises a processor; a memory having stored therein executable instructions of the processor; wherein the processor is configured to perform the steps of the chip verification method via execution of the executable instructions.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" platform.

An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 600 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 7, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 that connects the various system components (including the storage unit 620 and the processing unit 610), a display unit 640, and the like.

Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention described in the chip verification method section above in this specification. For example, the processing unit 610 may perform the steps as shown in fig. 1.

The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.

The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 630 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In the chip verification device, the program in the memory is executed by the processor to realize the steps of the chip verification method, so the device can also obtain the technical effect of the chip verification method.

The embodiment of the invention also provides a computer readable storage medium for storing a program, and the program realizes the steps of the chip verification method when being executed by a processor. In some possible embodiments, aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present invention described in the chip verification method section above of this specification when the program product is executed on the terminal device.

Referring to fig. 8, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be executed on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The program in the computer storage medium implements the steps of the chip verification method when executed by a processor, and therefore, the computer storage medium can also achieve the technical effects of the chip verification method.

The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.

Claims

1. A chip verification method is characterized by comprising the following steps:

generating excitation data according to the model configuration parameters;

2. The chip verification method according to claim 1, wherein the generating excitation data according to the model configuration parameters comprises:

3. The chip verification method according to claim 1, wherein the inputting the excitation data into the software reference model and the simulated hardware reference model, respectively, and obtaining a software reference output and a simulated hardware reference output, and the determining whether the simulated hardware reference model is evaluated by a reference model based on a comparison of the software reference output and the simulated hardware reference output comprises:

4. The chip verification method according to claim 1, wherein the performing simulation verification according to the excitation data and obtaining simulation output comprises:

5. The chip verification method according to claim 1, wherein the performing simulation verification according to the excitation data and obtaining simulation output comprises:

6. The chip verification method according to claim 1, wherein in response to the chip verification result being a verification failure, modifying the code of the simulation verification to perform a regression test;

and in response to the chip verification result being verification pass, saving the code of the simulation verification to perform regression testing.

7. The chip verification method according to claim 1, wherein the generating excitation data according to the model configuration parameters further comprises:

8. A chip verification system, comprising:

the reference model evaluation module is used for respectively inputting the excitation data into the software reference model and the simulated hardware reference model, acquiring a software reference output and a simulated hardware reference output, and determining whether the simulated hardware reference model passes the reference model evaluation or not based on the comparison between the software reference output and the simulated hardware reference output;

9. A chip verification apparatus, comprising:

a processor;

a memory having stored therein executable instructions of the processor;

wherein the processor is configured to perform the steps of the chip verification method of any one of claims 1 to 7 via execution of the executable instructions.

10. A computer-readable storage medium storing a program, wherein the program, when executed by a processor, implements the steps of the chip verification method of any one of claims 1 to 7.