CN117973508A - Memristor cross array modeling method and device based on neural network - Google Patents
Memristor cross array modeling method and device based on neural network Download PDFInfo
- Publication number
- CN117973508A CN117973508A CN202311789537.6A CN202311789537A CN117973508A CN 117973508 A CN117973508 A CN 117973508A CN 202311789537 A CN202311789537 A CN 202311789537A CN 117973508 A CN117973508 A CN 117973508A
- Authority
- CN
- China
- Prior art keywords
- vector
- array
- input
- memristor cross
- cross array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 47
- 239000013598 vector Substances 0.000 claims abstract description 235
- 239000011159 matrix material Substances 0.000 claims abstract description 135
- 238000012549 training Methods 0.000 claims abstract description 49
- 230000010354 integration Effects 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 abstract description 12
- 230000008859 change Effects 0.000 abstract description 5
- 230000006870 function Effects 0.000 description 15
- 230000006399 behavior Effects 0.000 description 12
- 238000004891 communication Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000001994 activation Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Complex Calculations (AREA)
Abstract
The application relates to a memristor cross array modeling method and device based on a neural network, wherein the method comprises the following steps: determining the array position and the array size of a target memristor cross array and the integral time of a preset analog-to-digital converter, generating an input matrix, an input vector and an actual output vector of the target memristor cross array, and further constructing a training data set; based on the full connection layer, the rectifying linear unit and the analog-to-digital converter, a memristor cross array model is constructed; and training a memristor cross array model by using a training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining a functional relation between an input matrix and the input vector and the intercept parameter and the slope parameter. Therefore, the problems that the conventional linear fitting method cannot process the change condition of related parameters, and the function relation among the corresponding parameters is difficult to be deduced directly in theory due to the fact that the nonideal characteristic on the memristor cross array is complex are solved.
Description
Technical Field
The application relates to the technical field of memristor cross array modeling, in particular to a memristor cross array modeling method and device based on a neural network.
Background
With the development of deep learning technology, artificial intelligence has wide prospects in the fields of industry and scientific research, and mental support is a necessary factor for developing and researching artificial intelligence. Due to the limitation of the traditional chip von neumann architecture, the power consumption problem of the artificial intelligence chip is increasingly remarkable, and the high-energy-efficiency and high-power chip also becomes one of the bottlenecks for restricting the artificial intelligence. The memristor-based memory-calculation integrated architecture chip is expected to solve the problem, and the memristor cross array can realize a matrix vector multiplication function, so that a neural network can be deployed on the memristor cross array. At present, the technology, devices, circuits, architectures and the like related to memristors are fully researched and developed.
However, because the memristor cross array performs analog computation, the binary digital computation is different from that of a modern computer, so that the operation result has a precision problem, and the non-ideal effect of devices and circuits further increases noise in the computation process, so that the on-chip reasoning precision of the neural network is greatly limited.
Currently, modeling methods based on linear fitting extract the slope and intercept of linear regression from matrix vector multiplication computation data of memristor cross arrays. The matrix multiplication calculation data of the memristor cross array comprises an input vector X, an input matrix W and an output vector Y, and a linear model is extracted according to the dataIs defined as the parameter (i.e. slope k and intercept b, where/>)Representing element multiplication) and predicting the array output corresponding to any input combination using the model. However, based on the principle of linear fitting, the method virtually presets the values of parameters k and b independent of the input vector X, the input matrix W. In the actual implementation process, various non-ideal factors in the memristor cross array, such as voltage drop and leakage current problems, are closely related to the input vector and the input matrix in specific values, namely parameters k and b are not constants irrelevant to X, W but are functions of X, W, and the linear fitting method ignores the non-ideal factors, so that a modeling method of the linear fitting often has larger errors.
In summary, the existing linear fitting method cannot handle the situation that the slope and the intercept b change along with the input vector X and the input matrix W, and because the nonideal characteristic on the memristor cross array is complex, it is difficult to directly derive the functional relationship between the corresponding parameters in theory, so that a large error exists in the modeling method of the linear fitting, and the problem needs to be solved.
Disclosure of Invention
The application provides a memristor cross array modeling method and device based on a neural network, which are used for solving the problems that the conventional linear fitting method cannot process the change condition of related parameters, and the function relationship among the corresponding parameters is difficult to be deduced directly from theory due to the fact that the nonideal characteristic on the memristor cross array is complex.
An embodiment of a first aspect of the present application provides a memristor cross array modeling method based on a neural network, including the following steps: determining an array position, an array size and an integral time of a preset analog-to-digital converter of a target memristor cross array, generating an input matrix, an input vector and an actual output vector of the target memristor cross array based on the array position, the array size and the integral time, and constructing a training data set according to the input matrix, the input vector and the actual output vector; based on a preset full-connection layer, a rectification linear unit and the analog-to-digital converter, constructing a memristor cross array model; and training the memristor cross array model by using the training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining the functional relationship between the input matrix and the input vector and the intercept parameter and the slope parameter to obtain a memristor cross array modeling result.
Optionally, in one embodiment of the present application, the generating the input matrix, the input vector, and the actual output vector of the target memristor crossbar array based on the array location, the array size, and the integration time includes: generating the input matrix and the input vector according to the array position and the array size; mapping the input matrix into the conductance of the target memristor cross array, and simultaneously inputting the input vector to a vector input end of the target memristor cross array; and performing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain an actual output vector of the target memristor cross array.
Optionally, in an embodiment of the present application, the constructing a memristor cross array model based on the preset full connection layer, the rectifying linear unit and the analog-to-digital converter includes: through a plurality of full connection layers and a plurality of rectification linear units, a memristor cross array network is constructed; and generating the memristor cross array model based on the memristor cross array network and in combination with the analog-to-digital converter.
Optionally, in one embodiment of the present application, the generating the intercept parameter vector and the slope parameter vector of the target memristor intersection array, and determining the input matrix and the functional relationship between the input vector and the intercept parameter and the slope parameter, includes: performing a preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result; generating the intercept parameter vector and the slope parameter vector based on the each input matrix, the each input vector, and the memristor cross array network; calculating an output vector predictor using the analog-to-digital converter based on the intercept parameter vector, the slope parameter vector, and the matrix vector multiplication result; and determining the input matrix and the input vector of the target memristor cross array, and the functional relation between the intercept parameter and the slope parameter according to the output vector predicted value.
An embodiment of a second aspect of the present application provides a memristor cross array modeling apparatus based on a neural network, including: the first determining module is used for determining the array position, the array size and the integral time of a preset analog-to-digital converter of a target memristor cross array, generating an input matrix, an input vector and an actual output vector of the target memristor cross array based on the array position, the array size and the integral time, and constructing a training data set according to the input matrix, the input vector and the actual output vector; the modeling module is used for constructing a memristor cross array model based on a preset full-connection layer, a rectification linear unit and the analog-to-digital converter; and the second determining module is used for training the memristor cross array model by using the training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, determining the functional relation between the input matrix and the input vector and the intercept parameter and the slope parameter, and obtaining a memristor cross array modeling result.
Optionally, in one embodiment of the present application, the first determining module includes: a first generating unit configured to generate the input matrix and the input vector according to the array position and the array size; the deployment unit is used for mapping the input matrix into the conductance of the target memristor cross array, and simultaneously inputting the input vector to a vector input end of the target memristor cross array; and the execution unit is used for executing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain an actual output vector of the target memristor cross array.
Optionally, in one embodiment of the present application, the modeling module includes: the construction unit is used for constructing a memristor cross array network through a plurality of full-connection layers and a plurality of rectification linear units; and the second generation unit is used for generating the memristor cross array model based on the memristor cross array network and combining the analog-to-digital converter.
Optionally, in one embodiment of the present application, the second determining module includes: the first computing unit is used for executing preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result; a third generating unit, configured to generate the intercept parameter vector and the slope parameter vector based on the each input matrix, the each input vector, and the memristor cross array network; a second calculation unit for calculating an output vector predicted value using the analog-to-digital converter based on the intercept parameter vector, the slope parameter vector, and the matrix vector multiplication result; and a result unit for determining the input matrix and the input vector of the target memristor cross array, and the functional relation between the intercept parameter and the slope parameter according to the output vector predicted value.
An embodiment of a third aspect of the present application provides an electronic device, including: the memory, the processor and the computer program stored on the memory and capable of running on the processor, the processor executes the program to realize the memristor cross array modeling method based on the neural network as described in the embodiment.
A fourth aspect of the present application provides a computer readable storage medium storing a computer program which when executed by a processor implements a neural network based memristor cross array modeling method as above.
Thus, embodiments of the present application have the following beneficial effects:
According to the embodiment of the application, the array position, the array size and the preset integral time of the analog-to-digital converter of the target memristor cross array can be determined, the input matrix, the input vector and the actual output vector of the target memristor cross array are generated based on the array position, the array size and the integral time, and a training data set is constructed according to the input matrix, the input vector and the actual output vector; based on a preset full-connection layer, a rectification linear unit and an analog-to-digital converter, constructing a memristor cross array model; and training a memristor cross array model by using a training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining a functional relation between an input matrix and the input vector and the intercept parameter and the slope parameter to obtain a memristor cross array modeling result. According to the application, the implicit rule in the data set is learned by utilizing the learning capability of the neural network, so that the output result of the memristor cross array is predicted according to the input vector and the matrix, and the behavior characteristic of the memristor cross array is simulated, so that a more accurate behavior model is established. Therefore, the problems that the conventional linear fitting method cannot process the change condition of related parameters, and the function relation among the corresponding parameters is difficult to be deduced directly in theory due to the fact that the nonideal characteristic on the memristor cross array is complex are solved.
Additional aspects and advantages of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flowchart of a memristor cross array modeling method based on a neural network provided in an embodiment of the present application;
FIG. 2 is a schematic diagram of a training data set construction flow according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a data acquisition process for memristor cross array matrix vector multiplication according to one embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a memristor cross array model architecture according to one embodiment of the present disclosure;
FIG. 5 is an example diagram of a neural network-based memristor cross array modeling apparatus in accordance with an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
The device comprises a 10-memristor cross array modeling device based on a neural network, a 100-first determining module, a 200-modeling module, a 300-second determining module, a 601-memory, a 602-processor and 603-communication interfaces.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present application and should not be construed as limiting the application.
The memristor cross array modeling method and device based on the neural network in the embodiment of the application are described below with reference to the accompanying drawings. In order to solve the problems mentioned in the background art, the application provides a memristor cross array modeling method based on a neural network, in the method, an input matrix, an input vector and an actual output vector of a target memristor cross array are generated by determining the array position, the array size and the integral time of a preset analog-to-digital converter of the target memristor cross array, and based on the array position, the array size and the integral time, and a training data set is constructed according to the input matrix, the input vector and the actual output vector; based on a preset full-connection layer, a rectification linear unit and an analog-to-digital converter, constructing a memristor cross array model; and training a memristor cross array model by using a training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining a functional relation between an input matrix and the input vector and the intercept parameter and the slope parameter to obtain a memristor cross array modeling result. According to the application, the implicit rule in the data set is learned by utilizing the learning capability of the neural network, so that the output result of the memristor cross array is predicted according to the input vector and the matrix, and the behavior characteristic of the memristor cross array is simulated, so that a more accurate behavior model is established. Therefore, the problems that the conventional linear fitting method cannot process the change condition of related parameters, and the function relation among the corresponding parameters is difficult to be deduced directly in theory due to the fact that the nonideal characteristic on the memristor cross array is complex are solved.
Specifically, fig. 1 is a flowchart of a memristor cross array modeling method based on a neural network provided by an embodiment of the present application.
As shown in fig. 1, the memristor cross array modeling method based on the neural network includes the following steps:
in step S101, an array position, an array size, and an integration time of a preset analog-to-digital converter of the target memristor cross array are determined, and based on the array position, the array size, and the integration time, an input matrix, an input vector, and an actual output vector of the target memristor cross array are generated, and a training data set is constructed according to the input matrix, the input vector, and the actual output vector.
According to the embodiment of the application, the input vector X, the input matrix W and the actual output vector Y Actual practice is that of of the target memristor cross array can be calculated according to the array position, the array size and the integration time of the analog-to-digital converter of the target memristor cross array, and a training data set is constructed according to the calculation result of the memristor cross array, namely the input matrix, the input vector and the actual output vector, so that a reliable data basis is provided for training of the memristor cross array model.
Optionally, in one embodiment of the present application, generating the input matrix, the input vector, and the actual output vector of the target memristor crossbar array based on the array location, the array size, and the integration time includes: generating an input matrix and an input vector according to the array position and the array size; mapping the input matrix into the conductance of the target memristor cross array, and simultaneously inputting an input vector into a vector input end of the target memristor cross array; and performing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain an actual output vector of the target memristor cross array.
Specifically, as shown in fig. 2, the process of constructing the training data set of the target memristor cross array in the embodiment of the present application is as follows:
1. The array position, size and integration time to be modeled are determined. The embodiment of the application can model the whole memristor cross array on the chip, can model the local array, and can be introduced for an m-row n-column array modeling example; in addition, in the matrix multiplication process, the output vector obtained by the memristor cross array calculation needs to be integrated and quantized by an analog-to-digital converter (also called an ADC or a/D, which is a circuit module for converting continuous analog quantity into discrete digital quantity), as shown in fig. 3, where the integration time of the analog-to-digital converter can be set manually, and different integration times can generate different outputs, so that a fixed integration time t needs to be selected during modeling;
2. Randomly generating an input matrix of m rows and n columns, and inputting the input matrix into a memristor cross array, namely mapping the input matrix to the conductance value of the memristor cross array, as shown in fig. 3;
3. randomly generating p m-dimensional input vectors which actually form a matrix of p rows and m columns and are input to vector input ports of the memristor cross array;
4. The integration time of the analog-to-digital converter is set as t, matrix vector multiplication is started to be calculated, and multiplication calculation results are acquired from a vector output end, and q pieces of input matrix data are required to be acquired in total, so q pieces of cyclic acquisition data are required, and each input matrix correspondingly acquires p pieces of input vector corresponding results, namely p pieces of output vector Y Actual practice is that of ;
5. repeating the steps, and changing a new input matrix to obtain matrix vector multiplication data corresponding to different input matrixes.
The above-mentioned steps produce p×q pieces of data in total, each piece of data including three portions of input vector X, input matrix W and actual output vector Y Actual practice is that of , so that the above-mentioned obtained data can be sorted and packed by means of conventional data processing and deep learning kits, such as NumPy (a Python-based data analysis and processing kit) and PyTorch (a Python-based deep learning kit), etc., so as to construct a training data set.
In step S102, a memristor cross array model is constructed based on a preset full connection layer, a rectifying linear unit and an analog-to-digital converter.
After the training data set is constructed, the embodiment of the application can further utilize the full-connection layer, the rectifying linear unit and the analog-to-digital converter to construct a memristor cross array model, so that a behavior level model is established for the memristor cross array, the behavior characteristics of the memristor chip under non-ideal characteristics can be understood, and the method can be further applied to an off-chip training frame of a neural network as a chip simulator, so that the robustness and on-chip reasoning precision of the model are improved.
Optionally, in one embodiment of the present application, constructing a memristor cross array model based on a preset full connection layer, a rectifying linear unit, and an analog-to-digital converter includes: through a plurality of full connection layers and a plurality of rectification linear units, a memristor cross array network is constructed; based on the memristor cross array network, an analog-to-digital converter is combined to generate a memristor cross array model.
It should be noted that, in the embodiment of the present application, a memristor cross array network may be constructed by using multiple FC (full-connected) layers and multiple ReLU (RECTIFIER LINEAR Unit) activation functions, and an analog-to-digital converter may be combined to construct a memristor cross array model, so as to construct a behavioral model for the memristor cross array.
In step S103, a memristor cross array model is trained by using a training data set, an intercept parameter vector and a slope parameter vector of a target memristor cross array are generated, and a functional relationship between an input matrix and the input vector and the intercept parameter and the slope parameter is determined, so as to obtain a memristor cross array modeling result.
After the memristor cross array model is built, further, the embodiment of the present application can find the function Y Prediction =f (X, W) through the memristor cross array model and the training data set to predict Y Actual practice is that of through Y Prediction .
Therefore, the embodiment of the application can design a data set by utilizing the calculation result of the memristor cross array, train the memristor cross array model, learn the implicit rule in the data set by utilizing the learning capability of the neural network, and further predict the output result of the memristor cross array according to the input vector and the matrix so as to simulate the behavior characteristic of the memristor cross array and determine the functional relation between the input matrix and the input vector and the intercept parameter and the slope parameter.
Optionally, in one embodiment of the present application, generating an intercept parameter vector and a slope parameter vector of the target memristor intersection array, and determining a functional relationship between the input matrix and the input vector and the intercept parameter and the slope parameter, includes: performing preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result; generating an intercept parameter vector and a slope parameter vector based on each input matrix, each input vector, and the memristor cross array network; calculating an output vector predicted value by using an analog-to-digital converter based on the intercept parameter vector, the slope parameter vector and the matrix vector multiplication result; and determining the input matrix and the functional relation between the input vector and the intercept parameter and the slope parameter of the target memristor cross array according to the output vector predicted value.
Specifically, as shown in fig. 4, the embodiment of the present application may take the input vector X and the input matrix W in the training data set constructed as described above as model input, and Y Actual practice is that of as the target value of the model output Y Prediction , to train the memristor cross array model, and specifically includes the following steps:
1. the input vector X and the input matrix W are divided into three paths, wherein one path performs multiplication operation on the input vector X and the input matrix W to obtain WX, and the other two paths are respectively processed by two groups of neural networks;
2. The first group of neural networks is formed by activating a plurality of FC layers and a plurality of ReLUs, the neural networks receive an input vector X and an input matrix W, an n-dimensional vector k is obtained after calculation of each layer, as shown in fig. 4, "matrix deformation and connection" are that the input vector X and the input matrix W are connected together, d in represents the input dimension of the neural network, d out represents the output dimension of the neural network (namely the dimension n of k and b here), and h 1 and h 2 represent the dimension of the middle layer of the neural network;
3. The second group of neural networks are composed of a plurality of FC layers and a plurality of ReLU activations, have the same structure as the first group of neural networks, receive an input vector X and an input matrix W, and obtain an n-dimensional vector b after calculation of each layer;
4. The XW obtained in the step 1, the k obtained in the step 2 and the b obtained in the step 3 are calculated to obtain Wherein/>Representing element multiplication;
5. For the one obtained in step 4 After processing by the analog-to-digital converter, the final output Y Prediction is obtained, wherein the setting of the analog-to-digital converter is consistent with the measuring range and the precision of the analog-to-digital converter in the actual array in FIG. 3;
6. And training a memristor cross array model by using a training data set, taking the mean square error between the output Y Prediction of the model and the target value Y Actual practice is that of as a loss function, and carrying out back propagation on the value of the loss function, so that the learning and updating of the neural network model weight can be carried out.
It should be noted that, besides the memristor cross array model, a person skilled in the art may select a currently mainstream deep learning framework according to practical situations, for example PyTorch or TensorFlow, which is used for defining and training the neural network model, and is not limited herein.
In the actual implementation process, for the input matrix W, if the matrix is large in scale, a compression function g (W) can be considered, and after compression, the data scale of g (W) is obviously smaller than that of the original W, so that the neural network processing is facilitated; then X, g (W) and XW are input together into the neural network model, where XW directly provides the result of the matrix vector multiplication, and W, g (W) and XW are processed by the two sets of neural networks in fig. 4 after being connected, and the prediction result Y Prediction can be obtained as well.
Therefore, the embodiment of the application can learn the data characteristics of the memristor cross array, namely the functional relation of the parameters k and b to the parameters W and X by utilizing the learning capacity of the memristor cross array model, thereby establishing a more accurate behavior model and effectively improving the accuracy of the prediction result.
According to the memristor cross array modeling method based on the neural network, an input matrix, an input vector and an actual output vector of a target memristor cross array are generated by determining the array position, the array size and the preset integral time of an analog-to-digital converter of the target memristor cross array, and based on the array position, the array size and the integral time, and a training data set is constructed according to the input matrix, the input vector and the actual output vector; based on a preset full-connection layer, a rectification linear unit and an analog-to-digital converter, constructing a memristor cross array model; and training a memristor cross array model by using a training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining a functional relation between an input matrix and the input vector and the intercept parameter and the slope parameter to obtain a memristor cross array modeling result. According to the application, the implicit rule in the data set is learned by utilizing the learning capability of the neural network, so that the output result of the memristor cross array is predicted according to the input vector and the matrix, and the behavior characteristic of the memristor cross array is simulated, so that a more accurate behavior model is established.
Next, a memristor cross array modeling apparatus based on a neural network according to an embodiment of the present application is described with reference to the accompanying drawings.
FIG. 5 is a block schematic diagram of a neural network-based memristor cross array modeling apparatus in accordance with an embodiment of the present application.
As shown in fig. 5, the memristor cross array modeling apparatus 10 based on the neural network includes: the first determination module 100, the modeling module 200, and the second determination module 300.
The first determining module 100 is configured to determine an array position, an array size, and an integration time of a preset analog-to-digital converter of the target memristor cross array, generate an input matrix, an input vector, and an actual output vector of the target memristor cross array based on the array position, the array size, and the integration time, and construct a training data set according to the input matrix, the input vector, and the actual output vector.
The modeling module 200 is configured to construct a memristor cross array model based on a preset full connection layer, a rectifying linear unit and an analog-to-digital converter.
The second determining module 300 is configured to train the memristor cross array model using the training data set, generate an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determine a functional relationship between the input matrix and the input vector and the intercept parameter and the slope parameter, so as to obtain a memristor cross array modeling result.
Optionally, in one embodiment of the present application, the first determining module 100 includes: the device comprises a first generation unit, a deployment unit and an execution unit.
The first generation unit is used for generating an input matrix and an input vector according to the array position and the array size.
And the deployment unit is used for mapping the input matrix into the conductance of the target memristor cross array and simultaneously inputting the input vector into the vector input end of the target memristor cross array.
And the execution unit is used for executing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain the actual output vector of the target memristor cross array.
Alternatively, in one embodiment of the application, the modeling module 200 includes: a construction unit and a second generation unit.
The construction unit is used for constructing a memristor cross array network through a plurality of full-connection layers and a plurality of rectification linear units.
The second generation unit is used for generating a memristor cross array model based on the memristor cross array network and combining with an analog-to-digital converter.
Optionally, in one embodiment of the present application, the second determining module 300 includes: the device comprises a first computing unit, a third generating unit, a second computing unit and a result unit.
The first calculation unit is used for performing preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result.
And a third generating unit for generating an intercept parameter vector and a slope parameter vector based on each input matrix, each input vector and the memristor cross array network.
And a second calculation unit for calculating an output vector predicted value by using the analog-to-digital converter based on the intercept parameter vector, the slope parameter vector and the matrix vector multiplication result.
And the result unit is used for determining the input matrix and the functional relation between the input vector and the intercept parameter and the slope parameter of the target memristor cross array according to the output vector predicted value.
It should be noted that the foregoing explanation of the embodiment of the memristor cross array modeling method based on the neural network is also applicable to the memristor cross array modeling device based on the neural network of the embodiment, and will not be repeated herein.
According to the memristor cross array modeling device based on the neural network, a first determining module is used for determining the array position, the array size and the integral time of a preset analog-to-digital converter of a target memristor cross array, generating an input matrix, an input vector and an actual output vector of the target memristor cross array based on the array position, the array size and the integral time, and constructing a training data set according to the input matrix, the input vector and the actual output vector; the modeling module is used for constructing a memristor cross array model based on a preset full-connection layer, a rectification linear unit and an analog-to-digital converter; the second determining module is used for training the memristor cross array model by using the training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, determining a functional relation between the input matrix and the input vector and the intercept parameter and the slope parameter, and obtaining a memristor cross array modeling result. According to the application, the implicit rule in the data set is learned by utilizing the learning capability of the neural network, so that the output result of the memristor cross array is predicted according to the input vector and the matrix, and the behavior characteristic of the memristor cross array is simulated, so that a more accurate behavior model is established.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device may include:
a memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602.
The processor 602 implements the memristor cross array modeling method based on the neural network provided in the above embodiment when executing a program.
Further, the electronic device further includes:
A communication interface 603 for communication between the memory 601 and the processor 602.
A memory 601 for storing a computer program executable on the processor 602.
The memory 601 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one disk memory.
If the memory 601, the processor 602, and the communication interface 603 are implemented independently, the communication interface 603, the memory 601, and the processor 602 may be connected to each other through a bus and perform communication with each other. The bus may be an industry standard architecture (Industry Standard Architecture, abbreviated ISA) bus, an external device interconnect (PERIPHERAL COMPONENT, abbreviated PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus.
Alternatively, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a chip, the memory 601, the processor 602, and the communication interface 603 may perform communication with each other through internal interfaces.
The processor 602 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the application.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, realizes the memristor cross array modeling method based on the neural network.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or N embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present application, "N" means at least two, for example, two, three, etc., unless specifically defined otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and additional implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order from that shown or discussed, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the embodiments of the present application.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or N wires, a portable computer cartridge (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It is to be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
Those of ordinary skill in the art will appreciate that all or a portion of the steps carried out in the method of the above-described embodiments may be implemented by a program to instruct related hardware, where the program may be stored in a computer readable storage medium, and where the program, when executed, includes one or a combination of the steps of the method embodiments.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing module, or each unit may exist alone physically, or two or more units may be integrated in one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules may also be stored in a computer readable storage medium if implemented in the form of software functional modules and sold or used as a stand-alone product.
The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, or the like. While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (10)
1. The memristor cross array modeling method based on the neural network is characterized by comprising the following steps of:
Determining an array position, an array size and an integral time of a preset analog-to-digital converter of a target memristor cross array, generating an input matrix, an input vector and an actual output vector of the target memristor cross array based on the array position, the array size and the integral time, and constructing a training data set according to the input matrix, the input vector and the actual output vector;
based on a preset full-connection layer, a rectification linear unit and the analog-to-digital converter, constructing a memristor cross array model;
And training the memristor cross array model by using the training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, and determining the functional relationship between the input matrix and the input vector and the intercept parameter and the slope parameter to obtain a memristor cross array modeling result.
2. The method of claim 1, wherein the generating the input matrix, input vector, and actual output vector of the target memristor crossbar array based on the array location, the array size, and the integration time comprises:
Generating the input matrix and the input vector according to the array position and the array size;
mapping the input matrix into the conductance of the target memristor cross array, and simultaneously inputting the input vector to a vector input end of the target memristor cross array;
And performing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain an actual output vector of the target memristor cross array.
3. The method of claim 2, wherein constructing a memristor cross array model based on the preset fully-connected layer, the rectifying linear unit, and the analog-to-digital converter comprises:
through a plurality of full connection layers and a plurality of rectification linear units, a memristor cross array network is constructed;
And generating the memristor cross array model based on the memristor cross array network and in combination with the analog-to-digital converter.
4. The method of claim 3, wherein the generating intercept parameter vectors and slope parameter vectors for the target memristor crossover array and determining the input matrix and the functional relationship between the input vectors and the intercept parameters and slope parameters comprises:
Performing a preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result;
generating the intercept parameter vector and the slope parameter vector based on the each input matrix, the each input vector, and the memristor cross array network;
calculating an output vector predictor using the analog-to-digital converter based on the intercept parameter vector, the slope parameter vector, and the matrix vector multiplication result;
And determining the input matrix and the input vector of the target memristor cross array, and the functional relation between the intercept parameter and the slope parameter according to the output vector predicted value.
5. A memristor cross array modeling apparatus based on a neural network, comprising:
The first determining module is used for determining the array position, the array size and the integral time of a preset analog-to-digital converter of a target memristor cross array, generating an input matrix, an input vector and an actual output vector of the target memristor cross array based on the array position, the array size and the integral time, and constructing a training data set according to the input matrix, the input vector and the actual output vector;
the modeling module is used for constructing a memristor cross array model based on a preset full-connection layer, a rectification linear unit and the analog-to-digital converter;
and the second determining module is used for training the memristor cross array model by using the training data set, generating an intercept parameter vector and a slope parameter vector of the target memristor cross array, determining the functional relation between the input matrix and the input vector and the intercept parameter and the slope parameter, and obtaining a memristor cross array modeling result.
6. The apparatus of claim 5, wherein the first determining module comprises:
A first generating unit configured to generate the input matrix and the input vector according to the array position and the array size;
The deployment unit is used for mapping the input matrix into the conductance of the target memristor cross array, and simultaneously inputting the input vector to a vector input end of the target memristor cross array;
And the execution unit is used for executing preset matrix vector multiplication operation based on the integration time, the input matrix and the input vector to obtain an actual output vector of the target memristor cross array.
7. The apparatus of claim 6, wherein the modeling module comprises:
The construction unit is used for constructing a memristor cross array network through a plurality of full-connection layers and a plurality of rectification linear units;
And the second generation unit is used for generating the memristor cross array model based on the memristor cross array network and combining the analog-to-digital converter.
8. The apparatus of claim 7, wherein the second determining module comprises:
the first computing unit is used for executing preset matrix vector multiplication operation on each input matrix and each input vector in the training data set to obtain a matrix vector multiplication result;
A third generating unit, configured to generate the intercept parameter vector and the slope parameter vector based on the each input matrix, the each input vector, and the memristor cross array network;
a second calculation unit for calculating an output vector predicted value using the analog-to-digital converter based on the intercept parameter vector, the slope parameter vector, and the matrix vector multiplication result;
And a result unit for determining the input matrix and the input vector of the target memristor cross array, and the functional relation between the intercept parameter and the slope parameter according to the output vector predicted value.
9. An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the neural network-based memristor cross-array modeling method of any one of claims 1-4.
10. A computer readable storage medium having stored thereon a computer program, wherein the program is executed by a processor for implementing the neural network-based memristor cross-array modeling method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311789537.6A CN117973508A (en) | 2023-12-22 | 2023-12-22 | Memristor cross array modeling method and device based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311789537.6A CN117973508A (en) | 2023-12-22 | 2023-12-22 | Memristor cross array modeling method and device based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117973508A true CN117973508A (en) | 2024-05-03 |
Family
ID=90852350
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311789537.6A Pending CN117973508A (en) | 2023-12-22 | 2023-12-22 | Memristor cross array modeling method and device based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117973508A (en) |
-
2023
- 2023-12-22 CN CN202311789537.6A patent/CN117973508A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109754066B (en) | Method and apparatus for generating a fixed-point neural network | |
| CN112434462B (en) | Method and equipment for obtaining model | |
| TWI780284B (en) | Systems and methods for mapping matrix calculations to a matrix multiply accelerator | |
| JP2023501230A (en) | Memristor-based neural network training method and its training device | |
| US20230196202A1 (en) | System and method for automatic building of learning machines using learning machines | |
| KR20160143548A (en) | Method and apparatus for automatic tuning of artificial neural networks | |
| CN110309911A (en) | Neural network model verification method, device, computer equipment and storage medium | |
| CN112101525A (en) | Method, device and system for designing neural network through NAS | |
| US12014130B2 (en) | System and method for ESL modeling of machine learning | |
| WO2019043990A1 (en) | Controller of array including neuromorphic element, method for calculating discretization step size, and program | |
| CN110046706A (en) | Model generating method, device and server | |
| CN114861890B (en) | Method and device for constructing neural network, computing equipment and storage medium | |
| CN109508784A (en) | A kind of design method of neural network activation primitive | |
| CN115204355A (en) | Neural processing unit capable of reusing data and method thereof | |
| CN109670582B (en) | Design method of full-fixed-point neural network | |
| CN115699028A (en) | Efficient tile mapping for line-by-line convolutional neural network mapping that simulates artificial intelligence network reasoning | |
| CN117973508A (en) | Memristor cross array modeling method and device based on neural network | |
| CN116011681A (en) | Meteorological data prediction method and device, storage medium and electronic device | |
| Wang et al. | Deep neural network mapping and performance analysis on tiled rram architecture | |
| JP7398625B2 (en) | Machine learning devices, information processing methods and programs | |
| Lu et al. | NVMLearn: a simulation platform for non-volatile-memory-based deep learning hardware | |
| CN114860548A (en) | Power consumption management method, system, device and medium for server | |
| Rosado-Muñoz et al. | An IP core and GUI for implementing multilayer perceptron with a fuzzy activation function on configurable logic devices | |
| Brassai et al. | Neural control based on RBF network implemented on FPGA | |
| CN116134415A (en) | Pulse generation for updating a crossbar array |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |