TWI791725B - Neural network operation method, integrated circuit chip device and related products - Google Patents
Neural network operation method, integrated circuit chip device and related products Download PDFInfo
- Publication number
- TWI791725B TWI791725B TW107147412A TW107147412A TWI791725B TW I791725 B TWI791725 B TW I791725B TW 107147412 A TW107147412 A TW 107147412A TW 107147412 A TW107147412 A TW 107147412A TW I791725 B TWI791725 B TW I791725B
- Authority
- TW
- Taiwan
- Prior art keywords
- data
- matrix
- processing circuit
- floating
- neural network
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Neurology (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Advance Control (AREA)
- Processing Of Color Television Signals (AREA)
- Image Processing (AREA)
Abstract
一種集成電路芯片裝置及相關產品，所述集成電路芯片裝置包括：所述集成電路芯片裝置用於執行神經網絡正向運算，所述神經網絡包含n層；所述集成電路芯片裝置包括：主處理電路、k個分支處理電路以及k組基礎處理電路，所述主處理電路與所述k個分支處理電路分別連接，k個分支處理電路中每個分支處理電路對應k組基礎處理電路中的一組基礎處理電路，所述一組基礎處理電路包括至少一個基礎處理電路；所述分支處理電路包括：數據類型運算電路，用於執行浮點類型數據與定點類型數據之間的轉換。 An integrated circuit chip device and related products, the integrated circuit chip device includes: the integrated circuit chip device is used to execute the forward operation of the neural network, the neural network includes n layers; the integrated circuit chip device includes: a main processing circuits, k branch processing circuits, and k groups of basic processing circuits, the main processing circuit is connected to the k branch processing circuits respectively, and each branch processing circuit in the k branch processing circuits corresponds to one of the k groups of basic processing circuits A set of basic processing circuits, the set of basic processing circuits includes at least one basic processing circuit; the branch processing circuit includes: a data type operation circuit for performing conversion between floating-point type data and fixed-point type data.
Description
本披露涉及神經網絡領域，尤其涉及一種集成電路芯片裝置及相關產品。 The present disclosure relates to the field of neural networks, in particular to an integrated circuit chip device and related products.
人工神經網絡(Artificial Neural Network，ANN)，是20世紀80年代以來人工智能領域興起的研究熱點。它從信息處理角度對人腦神經元網絡進行抽象，建立某種簡單模型，按不同的連接方式組成不同的網絡。在工程與學術界也常直接簡稱為神經網絡或類神經網絡。神經網絡是一種運算模型，由大量的節點(或稱神經元)之間相互聯接構成。現有的神經網絡的運算基於中央處理器(Central Processing Unit，CPU)或圖形處理器(Graphics Processing Unit，GPU)來實現神經網絡的運算，此種運算的計算量大，功耗高。 Artificial Neural Network (ANN) is a research hotspot in the field of artificial intelligence since the 1980s. It abstracts the human brain neuron network from the perspective of information processing, establishes a simple model, and forms different networks according to different connection methods. In engineering and academia, it is often referred to directly as a neural network or a neural network. A neural network is an operational model consisting of a large number of nodes (or neurons) connected to each other. The calculation of the existing neural network is based on a central processing unit (Central Processing Unit, CPU) or a graphics processing unit (Graphics Processing Unit, GPU) to realize the calculation of the neural network, which requires a large amount of calculation and high power consumption.
本披露實施例提供了一種集成電路芯片裝置及相關產品，可提升計算裝置的處理速度，提高效率。 Embodiments of the present disclosure provide an integrated circuit chip device and related products, which can increase the processing speed of a computing device and improve efficiency.
第一方面，提供一種集成電路芯片裝置，所述集成電路芯片裝置用於執行神經網絡正向運算，所述神經網絡包含n層；所述集成電路芯片裝置包括：主處理電路、k個分支處理電路以及k組基礎處理電路，所述主處理電路與所述k個分支處理電路分別連接，k個分支處理電路中每個分支處理電路對應k組基礎處理電路中的一組基礎處理電路，所述一組基礎處理電路包括至少一個基礎處理電路；所述分支處理電路包括：數據類型運算電路，用於執行浮點類型數據與定點類型數據之間的轉換；所述主處理電路，用於接收第一計算指令，解析第一計算指令得到所述第一計算指令在所述正向運算的第i層包含的第一運算指令以及第一計算指令對應的輸入數據以及權值數據，所述i的取值範圍為大於等於1小於等於n的整數，如所述i大於等於2，所述輸入數據為第i-1層的輸出數據；所述主處理電路，用於依據該輸入數據、權值數據以及第一運算指令確定第一運算指令的第一複雜度，依據所述第一複雜度確定所述第一運算指令對應的第一數據類型，依據所述第一複雜度確定是否向所述k個分支處理電路發送開啟指令開啟所述數據類型運算電路；所述第一數據類型為浮點數據類型或定點數據類型；所述主處理電路，還用於依據所述第一運算指令的類型將第一數據類型的所述輸入數據以及第一數據類型的所述權值數據劃分成廣播數據塊以及分發數據塊，對所述分發數據塊進行拆分處理得到多個基本數據塊，將所述多個基本數據塊分發至所述K個分支處理電路中的至少一分支處理電路，將所述廣播數據塊廣播至所述K個分支處理電路； 所述k個分支處理電路，用於在接收到開啟指令時，通過所述數據類型運算電路將廣播數據塊以及接收到的基本數據塊轉換成第一數據類型的廣播數據塊和第一數據類型的接收到的基本數據塊；將第一數據類型的廣播數據塊和定點類型的接收到的基本數據塊轉發給基礎處理電路；所述k組基礎處理電路，用於將第一數據類型的廣播數據塊和第一數據類型的接收到的基本數據塊以並行方式執行運算得到第一數據類型的運算結果，並將運算結果發送至所述k個分支處理電路；所述k個分支處理電路，用於通過所述數據類型運算電路將第一數據類型的運算結果轉換成第二數據類型的運算結果，將第二數據類型的運算結果發送至所述主處理電路；所述主處理電路，用於對所述第二數據類型的運算結果處理得到所述第一運算指令的指令結果完成第i層包含的第一運算指令運算。 In a first aspect, there is provided an integrated circuit chip device, the integrated circuit chip device is used to execute the forward operation of the neural network, and the neural network includes n layers; the integrated circuit chip device includes: a main processing circuit, k branch processing circuits and k groups of basic processing circuits, the main processing circuit is connected to the k branch processing circuits respectively, and each branch processing circuit in the k branch processing circuits corresponds to a group of basic processing circuits in the k group of basic processing circuits, so The set of basic processing circuits includes at least one basic processing circuit; the branch processing circuit includes: a data type operation circuit for performing conversion between floating-point type data and fixed-point type data; the main processing circuit is for receiving The first calculation instruction, analyzing the first calculation instruction to obtain the first calculation instruction contained in the first calculation instruction in the i-th layer of the forward operation, and the input data and weight data corresponding to the first calculation instruction, the i The range of values is an integer greater than or equal to 1 and less than or equal to n. For example, if i is greater than or equal to 2, the input data is the output data of the i-1th layer; the main processing circuit is used to The value data and the first operation instruction determine the first complexity of the first operation instruction, determine the first data type corresponding to the first operation instruction according to the first complexity, and determine whether to add the first data type to the first operation instruction according to the first complexity The k branch processing circuits send an open instruction to open the data type operation circuit; the first data type is a floating point data type or a fixed point data type; the main processing circuit is also used for according to the first operation instruction Divide the input data of the first data type and the weight data of the first data type into broadcast data blocks and distribution data blocks, and split the distribution data blocks to obtain a plurality of basic data blocks, distributing the plurality of basic data blocks to at least one of the K branch processing circuits, and broadcasting the broadcast data block to the K branch processing circuits; The k branch processing circuits are used to convert the broadcast data block and the received basic data block into the broadcast data block of the first data type and the first data type through the data type operation circuit when receiving the start instruction The received basic data blocks; the broadcast data blocks of the first data type and the received basic data blocks of the fixed-point type are forwarded to the basic processing circuit; the k groups of basic processing circuits are used to broadcast the first data type The data block and the received basic data block of the first data type perform operations in parallel to obtain an operation result of the first data type, and send the operation result to the k branch processing circuits; the k branch processing circuits, It is used to convert the operation result of the first data type into the operation result of the second data type through the data type operation circuit, and send the operation result of the second data type to the main processing circuit; the main processing circuit uses The instruction result of the first operation instruction is obtained by processing the operation result of the second data type to complete the operation of the first operation instruction included in the i-th layer.
第二方面，提供一種神經網絡運算裝置，所述神經網絡運算裝置包括一個或多個第一方面提供的集成電路芯片裝置。 In a second aspect, a neural network computing device is provided, and the neural network computing device includes one or more integrated circuit chip devices provided in the first aspect.
第三方面，提供一種組合處理裝置，所述組合處理裝置包括：第二方面提供的神經網絡運算裝置、通用互聯介面和通用處理裝置；所述神經網絡運算裝置通過所述通用互聯介面與所述通用處理裝置連接。 A third aspect provides a combined processing device, which includes: the neural network computing device provided in the second aspect, a general interconnection interface, and a general processing device; the neural network computing device communicates with the Universal processing device connection.
第四方面，提供一種芯片，所述芯片集成第一方面的裝置、第二方面的裝置或第三方面的裝置。 In a fourth aspect, a chip is provided, and the chip integrates the device of the first aspect, the device of the second aspect, or the device of the third aspect.
第五方面，提供一種電子設備，所述電子設備包括第四方面的芯片。 In a fifth aspect, an electronic device is provided, and the electronic device includes the chip of the fourth aspect.
第六方面，提供一種神經網絡的運算方法，所述方法應用在集成電路芯片裝置內，所述集成電路芯片裝置包括：第一方面所述的集成電路芯片裝置，所述集成電路芯片裝置用於執行神經網絡的正向運算。 In a sixth aspect, there is provided a method for calculating a neural network, the method is applied in an integrated circuit chip device, and the integrated circuit chip device includes: the integrated circuit chip device described in the first aspect, and the integrated circuit chip device is used for Performs the forward operation of the neural network.
可以看出，通過本披露實施例，提供數據轉換運算電路將數據塊的類型進行轉換後運算，節省了傳輸資源以及計算資源，所以其具有功耗低，計算量小的優點。 It can be seen that, through the embodiment of the present disclosure, the data conversion operation circuit is provided to perform operation after conversion of the type of the data block, which saves transmission resources and computing resources, so it has the advantages of low power consumption and small amount of calculation.
A、B:矩陣 A, B: Matrix
P:向量 P: vector
H:特徵圖的高 H: the height of the feature map
W:特徵圖的寬 W: the width of the feature map
C:通道數量 C: Number of channels
KH:卷積核的高 KH: the height of the convolution kernel
KW:卷積核的寬 KW: the width of the convolution kernel
M、L、N:行/列數 M, L, N: number of rows/columns
S401b、S402b、S403b、S401、S402、S403、S404:步驟 S401b, S402b, S403b, S401, S402, S403, S404: steps
10:神經網絡處理器板卡 10: Neural network processor board
11:神經網絡芯片封裝結構 11: Neural network chip packaging structure
12:第一電氣及非電氣連接裝置 12: The first electrical and non-electrical connection device
13:第一基板 13: The first substrate
111、21:神經網絡芯片 111, 21: Neural network chip
112:第二電氣及非電氣連接裝置 112: Second electrical and non-electrical connection device
113、24:第二基板 113, 24: second substrate
22:焊盤 22:Pad
23:焊球 23: solder ball
25:第二基板24上的連接點
25: connection point on the
26:引腳 26: pin
27:絕緣填充物 27: Insulation filler
28:散熱膏 28: thermal paste
29:金屬外殼散熱片 29: metal shell heat sink
圖1a是一種神經網絡的正向運算示意圖。 Fig. 1a is a schematic diagram of forward operation of a neural network.
圖1b為一種定點數據類型的示意結構圖。 Fig. 1b is a schematic structural diagram of a fixed-point data type.
圖2a為卷積輸入數據示意圖。 Figure 2a is a schematic diagram of convolution input data.
圖2b為卷積核示意圖。 Figure 2b is a schematic diagram of a convolution kernel.
圖2c為輸入數據的一個三維數據塊的運算窗口示意圖。 Fig. 2c is a schematic diagram of an operation window of a three-dimensional data block of input data.
圖2d為輸入數據的一個三維數據塊的另一運算窗口示意圖。 Fig. 2d is a schematic diagram of another operation window of a three-dimensional data block of input data.
圖2e為輸入數據的一個三維數據塊的又一運算窗口示意圖. Fig. 2e is a schematic diagram of another operation window of a three-dimensional data block of input data.
圖3是一種神經網絡芯片的結構示意圖。 Fig. 3 is a schematic structural diagram of a neural network chip.
圖4a為矩陣乘以矩陣示意圖。 Figure 4a is a schematic diagram of matrix-by-matrix multiplication.
圖4b為矩陣乘以矩陣的方法流程圖。 Fig. 4b is a flowchart of a method for multiplying a matrix by a matrix.
圖4c為矩陣乘以向量示意圖。 Fig. 4c is a schematic diagram of matrix multiplication by vector.
圖4d為矩陣乘以向量的方法流程圖。 Fig. 4d is a flowchart of a method for multiplying a matrix by a vector.
圖5a為本披露還揭露了一個組合處理裝置結構示意圖。 Fig. 5a is a schematic structural diagram of a combined processing device disclosed in this disclosure.
圖5b為本披露還揭露了一個組合處理裝置另一種結構示意圖。 Fig. 5b is a schematic diagram of another structure of a combined processing device disclosed in this disclosure.
圖5c為本披露實施例提供的一種神經網絡處理器板卡的結構示意圖；圖5d為本披露實施例流提供的一種神經網絡芯片封裝結構的結構示意圖；圖6為本披露實施例流提供的一種神經網絡芯片封裝結構的示意圖；圖6a為本披露實施例流提供的另一種神經網絡芯片封裝結構的示意圖。 Figure 5c is a schematic structural diagram of a neural network processor board provided by an embodiment of this disclosure; Figure 5d is a schematic structural diagram of a neural network chip packaging structure provided by an embodiment of this disclosure; Figure 6 is a schematic structural diagram of a neural network chip package provided by an embodiment of this disclosure A schematic diagram of a neural network chip packaging structure; FIG. 6a is a schematic diagram of another neural network chip packaging structure provided by an embodiment of the present disclosure.
為了使本技術領域的人員更好地理解本披露方案，下面將結合本披露實施例中的圖式，對本披露實施例中的技術方案進行清楚、完整地描述，顯然，所描述的實施例僅僅是本披露一部分實施例，而不是全部的實施例。基於本披露中的實施例，所屬技術領域中具有通常知識者在沒有作出創造性勞動前提下所獲得的所有其他實施例，都屬於本披露保護的範圍。 In order to enable those skilled in the art to better understand the present disclosure, the following will clearly and completely describe the technical solutions in the present disclosure embodiments in conjunction with the drawings in the present disclosure embodiments. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. Based on the embodiments in the present disclosure, all other embodiments obtained by persons with ordinary knowledge in the technical field without creative efforts fall within the protection scope of the present disclosure.
在第一方面提供的裝置中，所述主處理電路，具體用於將所述第一複雜度與預設閾值比較，如所述第一複雜度高於所述預設閾值，確定所述第一數據類型為定點類型，如所述第一複雜度低於或等於所述預設閾值，確定所述第一數據類型為浮點類型。 In the device provided in the first aspect, the main processing circuit is specifically configured to compare the first complexity with a preset threshold, and determine if the first complexity is higher than the preset threshold A data type is a fixed-point type, and if the first complexity is lower than or equal to the preset threshold, it is determined that the first data type is a floating-point type.
在第一方面提供的裝置中，所述主處理電路，具體用於確定所述輸入數據以及權值數據屬於第二數據類型，如所述第二數據類型與所述第一 數據類型不同，向所述k個分支處理電路發送開啟指令，所述開啟指令用於開啟所述分支處理電路包括的數據類型運算電路。 In the device provided in the first aspect, the main processing circuit is specifically configured to determine that the input data and the weight data belong to the second data type, such as the second data type being the same as the first If the data types are different, an enabling instruction is sent to the k branch processing circuits, and the enabling instruction is used to enable a data type operation circuit included in the branch processing circuits.
在第一方面提供的裝置中，所述主處理電路，具體用於如所述第一運算指令為卷積運算指令，所述輸入數據為卷積輸入數據，所述權值數據為卷積核，第一複雜度=α*C*kW*kW*M*N*W*C*H；其中，α為卷積系數，取值範圍為大於1；C、kW、kW、M為卷積核四個維度的值，N、W、C、H為卷積輸入數據四個維度的值；如所述第一複雜度大於設定閾值，確定該卷積輸入數據以及卷積核是否為浮點數據，如該卷積輸入數據以及卷積核不為浮點數據，通知所述k個分支處理電路將該卷積輸入數據轉換成浮點數據，將卷積核轉換成浮點數據，然後將卷積輸入數據、卷積核以浮點數據類型執行卷積運算。 In the device provided in the first aspect, the main processing circuit is specifically configured such that the first operation instruction is a convolution operation instruction, the input data is convolution input data, and the weight data is a convolution kernel , the first complexity=α*C*kW*kW*M*N*W*C*H; among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kW, M are convolution kernels The values of the four dimensions, N, W, C, and H are the values of the four dimensions of the convolution input data; if the first complexity is greater than the set threshold, determine whether the convolution input data and the convolution kernel are floating point numbers According to the data, if the convolution input data and the convolution kernel are not floating-point data, notify the k branch processing circuits to convert the convolution input data into floating-point data, and convert the convolution kernel into floating-point data. , and then the convolution input data, the convolution kernel performs the convolution operation in the floating-point data type.
在第一方面提供的裝置中，所述主處理電路，具體用於如所述第一運算指令為：矩陣乘矩陣運算指令，所述輸入數據為所述矩陣乘矩陣運算的第一矩陣，所述權值為所述矩陣乘矩陣運算的第二矩陣；第一複雜度=β*F*G*E*F；其中，β為矩陣系數，取值範圍為大於等於1，F、G為第一矩陣的行、列值，E、F為第二矩陣的行、列值；如所述第一複雜度大於設定閾值，確定該第一矩陣以及第二矩陣是否為浮點數據，如該第一矩陣以及第二矩陣不為浮點數據，通知所述k個分支處理電路將該第一矩陣轉換成浮點數據，將第二矩陣轉換成浮點數據，然後將第一矩陣、第二矩陣以浮點數據類型執行矩陣乘矩陣運算。 In the device provided in the first aspect, the main processing circuit is specifically configured such that the first operation instruction is: a matrix multiplication matrix operation instruction, and the input data is the first matrix of the matrix multiplication matrix operation, so The weight is the second matrix of the matrix multiplication matrix operation; the first complexity=β*F*G*E*F; wherein, β is a matrix coefficient, and the value range is greater than or equal to 1, and F and G are the first The row and column values of a matrix, E and F are the row and column values of the second matrix; if the first complexity is greater than the set threshold, it is determined whether the first matrix and the second matrix are floating-point data, such as the The first matrix and the second matrix are not floating-point data, and the k branch processing circuits are notified to convert the first matrix into floating-point data, convert the second matrix into floating-point data, and then convert the first matrix into floating-point data , and the second matrix performs a matrix-by-matrix operation in a floating-point data type.
在第一方面提供的裝置中，所述主處理電路，具體用於如所述第一運算指令為：矩陣乘向量運算指令，所述輸入數據為所述矩陣乘向量運算的第一矩陣，所述權值為所述矩陣乘向量運算的向量；第一複雜度=β*F*G*F；其中，β為矩陣系數，取值範圍為大於等於1，F、G為第一矩陣的行、列值，F為向量的列值；如所述第一複雜度大於設定閾值，確定該第一矩陣以及向量是否為浮點數據，如該第一矩陣以及向量不為浮點數據，將該第一矩陣轉換成浮點數據，通知所述k個分支處理電路將向量轉換成浮點數據，然後將第一矩陣、向量以浮點數據類型執行矩陣乘向量運算。 In the device provided in the first aspect, the main processing circuit is specifically configured such that the first operation instruction is: a matrix multiplication vector operation instruction, and the input data is the first matrix of the matrix multiplication vector operation, so The weight is the vector of the matrix multiplication vector operation; the first complexity=β*F*G*F; wherein, β is a matrix coefficient, and the value range is greater than or equal to 1, and F and G are rows of the first matrix , column value, F is the column value of the vector; if the first complexity is greater than the set threshold, determine whether the first matrix and the vector are floating-point data, if the first matrix and the vector are not floating-point data, converting the first matrix into floating-point data, instructing the k branch processing circuits to convert the vector into floating-point data, and then performing matrix multiplication and vector operation on the first matrix and vector in floating-point data type.
在第一方面提供的裝置中，所述主處理電路，具體用於如所述第一運算指令的類型為乘法指令，確定所述輸入數據為分發數據塊，所述權值數據為廣播數據塊；如所述第一運算指令的類型為卷積指令，確定所述輸入數據為廣播數據塊，所述權值數據為分發數據塊。 In the device provided in the first aspect, the main processing circuit is specifically configured to determine that the input data is a distribution data block and the weight data is a broadcast data block if the type of the first operation instruction is a multiplication instruction ; If the type of the first operation instruction is a convolution instruction, determine that the input data is a broadcast data block, and the weight data is a distribution data block.
在第一方面提供的裝置中，所述第i層還包括：偏置運算、全連接運算、通用矩陣乘法(General Matrix Multiplication，GEMM)運算、通用矩陣向量乘法(General Matrix Vector Multiplication，GEMV)運算、激活運算中的一種或任意組合。 In the device provided in the first aspect, the i-th layer further includes: offset operation, full connection operation, general matrix multiplication (General Matrix Multiplication, GEMM) operation, general matrix vector multiplication (General Matrix Vector Multiplication, GEMV) operation , one or any combination of activation operations.
在第一方面提供的裝置中，所述主處理電路包括：主寄存器或主片上緩存電路；所述基礎處理電路包括：基本寄存器或基本片上緩存電路。 In the device provided in the first aspect, the main processing circuit includes: a main register or a main on-chip cache circuit; the basic processing circuit includes: a basic register or a basic on-chip cache circuit.
在第一方面提供的裝置中，所述主處理電路包括：向量運算器電路、算數邏輯單元電路、累加器電路、矩陣轉置電路、直接內存存取電路或數據重排電路中的一種或任意組合。 In the device provided in the first aspect, the main processing circuit includes: one or any of a vector operator circuit, an arithmetic logic unit circuit, an accumulator circuit, a matrix transposition circuit, a direct memory access circuit or a data rearrangement circuit combination.
在第一方面提供的裝置中，所述輸入數據為：向量、矩陣、三維數據塊、四維數據塊以及n維數據塊中一種或任意組合；所述權值數據為：向量、矩陣、三維數據塊、四維數據塊以及n維數據塊中一種或任意組合。 In the device provided in the first aspect, the input data is: one or any combination of vectors, matrices, three-dimensional data blocks, four-dimensional data blocks, and n-dimensional data blocks; the weight data is: vectors, matrices, three-dimensional data One or any combination of blocks, four-dimensional data blocks, and n-dimensional data blocks.
如圖3所示，為本披露提供的一種集成電路芯片裝置，所述集成電路芯片裝置用於執行神經網絡正向運算，所述神經網絡包含n層；所述集成電路芯片裝置包括：主處理電路、k個分支處理電路以及k組基礎處理電路，所述主處理電路與所述k個分支處理電路分別連接，k個分支處理電路中每個分支處理電路對應k組基礎處理電路中的一組基礎處理電路，所述一組基礎處理電路包括至少一個基礎處理電路；所述分支處理電路包括：數據類型運算電路，用於執行浮點類型數據與定點類型數據之間的轉換；所述主處理電路，用於接收第一計算指令，解析第一計算指令得到所述第一計算指令在所述正向運算的第i層包含的第一運算指令以及第一計算指令對應的輸入數據以及權值數據，所述i的取值範圍為大於等於1小於等於n的整數，如所述i大於等於2，所述輸入數據為第i-1層的輸出數據；所述主處理電路，用於依據該輸入數據、權值數據以及第一運算指令確定第一運算指令的第一複雜度，依據所述第一複雜度確定所述第一運算指令對應的第一數據類型，依據所述第一複雜度確定是否向所述k個分支處 理電路發送開啟指令開啟所述數據類型運算電路；所述第一數據類型為浮點數據類型或定點數據類型；所述主處理電路，還用於依據所述第一運算指令的類型將第一數據類型的所述輸入數據以及第一數據類型的所述權值數據劃分成廣播數據塊以及分發數據塊，對所述分發數據塊進行拆分處理得到多個基本數據塊，將所述多個基本數據塊分發至所述K個分支處理電路中的至少一分支處理電路，將所述廣播數據塊廣播至所述K個分支處理電路；所述k個分支處理電路，用於在接收到開啟指令時，通過所述數據類型運算電路將廣播數據塊以及接收到的基本數據塊轉換成第一數據類型的廣播數據塊和第一數據類型的接收到的基本數據塊；將第一數據類型的廣播數據塊和定點類型的接收到的基本數據塊轉發給基礎處理電路；所述k組基礎處理電路，用於將第一數據類型的廣播數據塊和第一數據類型的接收到的基本數據塊以並行方式執行運算得到第一數據類型的運算結果，並將運算結果發送至所述k個分支處理電路；所述k個分支處理電路，用於通過所述數據類型運算電路將第一數據類型的運算結果轉換成第二數據類型的運算結果，將第二數據類型的運算結果發送至所述主處理電路；所述主處理電路，用於對所述第二數據類型的運算結果處理得到所述第一運算指令的指令結果完成第i層包含的第一運算指令運算。 As shown in FIG. 3 , it is an integrated circuit chip device provided by the present disclosure. The integrated circuit chip device is used to execute the forward operation of the neural network, and the neural network includes n layers; the integrated circuit chip device includes: main processing circuits, k branch processing circuits, and k groups of basic processing circuits, the main processing circuit is connected to the k branch processing circuits respectively, and each branch processing circuit in the k branch processing circuits corresponds to one of the k groups of basic processing circuits A group of basic processing circuits, the group of basic processing circuits includes at least one basic processing circuit; the branch processing circuit includes: a data type operation circuit for performing conversion between floating-point type data and fixed-point type data; the main A processing circuit, configured to receive a first computing instruction, analyze the first computing instruction to obtain the first computing instruction contained in the first computing instruction in the i-th layer of the forward operation, and the input data corresponding to the first computing instruction and the weight Value data, the value range of the i is an integer greater than or equal to 1 and less than or equal to n, if the i is greater than or equal to 2, the input data is the output data of the i-1th layer; the main processing circuit is used for Determine the first complexity of the first operation instruction according to the input data, the weight data, and the first operation instruction, determine the first data type corresponding to the first operation instruction according to the first complexity, and determine the first data type corresponding to the first operation instruction according to the first The complexity determines whether to the k branches The processing circuit sends an opening instruction to open the data type operation circuit; the first data type is a floating point data type or a fixed point data type; the main processing circuit is also used to convert the second type according to the type of the first operation instruction The input data of a data type and the weight data of the first data type are divided into a broadcast data block and a distribution data block, and the distribution data block is split to obtain multiple basic data blocks, and the multiple The basic data blocks are distributed to at least one branch processing circuit in the K branch processing circuits, and the broadcast data block is broadcast to the K branch processing circuits; the k branch processing circuits are used to receive When the instruction is turned on, the broadcast data block and the received basic data block are converted into the broadcast data block of the first data type and the received basic data block of the first data type through the data type operation circuit; The broadcast data blocks of the fixed-point type and the received basic data blocks of the fixed-point type are forwarded to the basic processing circuit; The blocks perform operations in parallel to obtain the operation results of the first data type, and send the operation results to the k branch processing circuits; the k branch processing circuits are used to convert the first data type through the data type operation circuit The operation result of the second data type is converted into the operation result of the second data type, and the operation result of the second data type is sent to the main processing circuit; the main processing circuit is used to process the operation result of the second data type to obtain The instruction result of the first operation instruction completes the operation of the first operation instruction contained in the i-th layer.
如圖1a所示，為本披露實施例提供的一種神經網絡的正向運算，每一層使用自己的輸入數據和權值按照層的類型所指定的運算規則計算得到相應的輸出數據； 神經網絡的正向運算過程(也叫推理，inference)是逐層處理各層的輸入數據，經過一定的計算，得到輸出數據的過程，具有如下特徵：某一層的輸入：某一層的輸入可以是神經網絡的輸入數據；某一層的輸入可以是其他層的輸出；某一層的輸入可以是本層上一時刻的輸出(對應於循環神經網絡的情況)；某一層可以同時從多個上述輸入源獲取輸入；某一層的輸出：某一層的輸出可以作為神經網絡的輸出結果；某一層的輸出可以是其它層的輸入；某一層的輸出可以是下一時刻本層的輸入(循環神經網絡的情況)；某一層的輸出可以向上述多個輸出方向輸出結果；具體地，所述神經網絡中的層的運算的類型包括但不限於以下幾種：卷積層(即執行卷積運算)；全連接層(即執行全連接運算)；歸一化(規則化)層：包括局部響應歸一化(Local Response Normalization，LRN)層，批標準化(Batch Normalization，BN)層等類型；池化層； 激活層：包括但不限於以下類型Sigmoid層，ReLU層，PReLu層，LeakyReLu層，Tanh層；層的反向運算，每一層的反向運算需要執行兩部分運算：一部分是使用可能是稀疏表示的輸出數據梯度和可能是稀疏表示的輸入數據計算出權值的梯度(用於在「權值更新」步驟更新本層的權值)，另一部分是使用可能是稀疏表示的輸出數據梯度和可能是稀疏表示的權值，計算出輸入數據梯度(用於作為反向運算中下一層的輸出數據梯度以供其進行反向運算)；反向運算按照與正向運算相反的順序，從最後一層開始反向傳遞梯度。 As shown in Figure 1a, it is a forward operation of a neural network provided by the embodiment of the present disclosure. Each layer uses its own input data and weights to calculate corresponding output data according to the operation rules specified by the layer type; The forward operation process (also called reasoning, inference) of the neural network is the process of processing the input data of each layer layer by layer, and after a certain calculation, the process of obtaining the output data has the following characteristics: the input of a certain layer: the input of a certain layer can be neural network The input data of the network; the input of a certain layer can be the output of other layers; the input of a certain layer can be the output of the previous moment of this layer (corresponding to the situation of the cyclic neural network); a certain layer can be obtained from multiple above-mentioned input sources at the same time Input; output of a certain layer: the output of a certain layer can be used as the output of the neural network; the output of a certain layer can be the input of other layers; the output of a certain layer can be the input of the current layer at the next moment (in the case of cyclic neural network) The output of a certain layer can output results to the above-mentioned multiple output directions; specifically, the types of operations of the layers in the neural network include but are not limited to the following: convolution layer (i.e., perform convolution operation); fully connected layer (i.e. perform fully connected operations); normalization (regularization) layer: including local response normalization (Local Response Normalization, LRN) layer, batch normalization (Batch Normalization, BN) layer and other types; pooling layer; Activation layer: including but not limited to the following types of Sigmoid layer, ReLU layer, PReLu layer, LeakyReLu layer, Tanh layer; the reverse operation of the layer, the reverse operation of each layer needs to perform two parts of the operation: one part is to use possibly sparse representation The gradient of the output data and the input data that may be sparsely represented calculate the gradient of the weight (used to update the weight of this layer in the "weight update" step), and the other part is to use the gradient of the output data that may be sparsely represented and may be The weight of the sparse representation calculates the input data gradient (used as the output data gradient of the next layer in the reverse operation for its reverse operation); the reverse operation is in the reverse order of the forward operation, starting from the last layer Pass the gradient back.
在一種可選方案中，某一層反向計算得到的輸出數據梯度可以來自：神經網絡最後的損失函數(lost function或者cost function)回傳的梯度；其它層的輸入數據梯度；本層上一時刻的輸入數據梯度(對應於循環神經網絡的情況)；某一層可以同時從多個上述源獲取輸出數據梯度；在執行完神經網絡的反向運算之後，就計算出了各層的權值的梯度，在這個步驟中，所述裝置的第一輸入緩存和第二輸入緩存分別用於存儲本層的權值和權值的梯度，然後在運算單元中使用權值梯度對權值進行更新；上文中提到的運算都是神經網絡中的一層的運算，對於多層神經網絡，其實現過程是，在正向運算中，當上一層人工神經網絡執行完成之後，下一層的運算指令會將運算單元中計算出的輸出數據作為下一層的輸入數據進 行運算(或者是對該輸出數據進行某些操作再作為下一層的輸入數據)，同時，將權值也替換為下一層的權值；在反向運算中，當上一層人工神經網絡的反向運算執行完成後，下一層運算指令會將運算單元中計算出的輸入數據梯度作為下一層的輸出數據梯度進行運算(或者是對該輸入數據梯度進行某些操作再作為下一層的輸出數據梯度)，同時將權值替換為下一層的權值；(用以下的圖表示，以下圖中虛線的箭頭表示反向運算，實線的箭頭表示正向運算，各圖下面的標注表示圖的含義) In an optional solution, the output data gradient obtained by reverse calculation of a certain layer can come from: the gradient returned by the last loss function (lost function or cost function) of the neural network; the input data gradient of other layers; the previous moment of this layer The input data gradient (corresponding to the situation of the cyclic neural network); a certain layer can obtain the output data gradient from multiple sources at the same time; after performing the reverse operation of the neural network, the gradient of the weight of each layer is calculated. In this step, the first input buffer and the second input buffer of the device are respectively used to store the weight and the gradient of the weight of this layer, and then use the gradient of the weight in the computing unit to update the weight; above The operations mentioned above are the operations of one layer of the neural network. For a multi-layer neural network, the implementation process is that in the forward operation, after the execution of the previous layer of artificial neural network is completed, the operation instruction of the next layer will be in the operation unit. The calculated output data is used as the input data of the next layer into the row operation (or perform certain operations on the output data and then use it as the input data of the next layer), and at the same time, replace the weight with the weight of the next layer; in the reverse operation, when the reverse of the previous layer of artificial neural network After the execution of the direction operation is completed, the next layer of operation instructions will use the input data gradient calculated in the operation unit as the output data gradient of the next layer for operation (or perform some operations on the input data gradient and then use it as the output data gradient of the next layer ), and replace the weight with the weight of the next layer; (represented by the following figure, the dotted arrow in the figure below indicates the reverse operation, the solid line arrow indicates the forward operation, and the labels below each figure indicate the meaning of the figure )
定點化數據的表示方法 Representation method of fixed-point data
定點化的方法是指將網絡中的某個數據塊的數據表示轉換成特定的某種固定小數點位置的數據表示方式(映射到電路裝置上數據的0/1比特位擺放方式)；在一種可選方案中，將多個數據組成個數據塊作為一個整體使用同樣的定點表示方法進行定點化表示；圖1b示出了根據本發明實施例的用於存儲數據的短位數定點數據結構的具體表示方法。其中，對於X位定點數據表示，1Bit位用於表示符號，M位用於表示整數部分，N位用於表示小數部分，其中M+N=X-1；相比於32位浮點數據表示形式，本發明採用的短位定點數據表示形式除了佔用比特位數更少外，對於神經網絡中同一層、同一類型的數據，如第一個卷積層的所有權值數據，還另外設置了一個標誌位Point location記錄小數點的位置，這樣可以根據實際數據的分布調整數據表示的精度與可表示數據範圍。 The fixed-point method refers to converting the data representation of a certain data block in the network into a specific data representation of a certain fixed decimal point position (mapped to the 0/1 bit arrangement of data on the circuit device); In an alternative solution, a plurality of data is formed into a data block as a whole to perform fixed-point representation using the same fixed-point representation method; FIG. 1 b shows a short-digit fixed-point data structure for storing data according to an embodiment of the present invention specific representation method. Among them, for X-bit fixed-point data representation, 1Bit is used to represent the sign, M-bit is used to represent the integer part, and N-bit is used to represent the fractional part, where M+N=X-1; compared to 32-bit floating-point data Representation form, the short-bit fixed-point data representation form adopted by the present invention not only occupies fewer bits, but also sets a The flag bit Point location records the position of the decimal point, so that the precision of data representation and the range of representable data can be adjusted according to the distribution of actual data.
對於浮點數的表示即32bit來表示，但是對於此技術方案，其採用定點數可以減少一個數值的比特位的位數，從而降低傳輸的數據量以及運算的數據量。 The representation of floating point numbers is represented by 32 bits, but for this technical solution, the use of fixed point numbers can reduce the number of bits of a numerical value, thereby reducing the amount of transmitted data and the amount of calculated data.
輸入數據用圖2a表示(N個樣本，每個樣本有C個通道，每個通道的特徵圖的高為H，寬為W)，權值也即卷積核用圖2b表示(有M個卷積核，每個卷積核有C個通道，高和寬分別為KH和KW)。對於輸入數據的N個樣本，卷積運算的規則都是一樣的，下面解釋在一個樣本上進行卷積運算的過程，在一個樣本上，M個卷積核中的每一個都要進行同樣的運算，每個卷積核運算得到一張平面特徵圖，M個卷積核最終計算得到M個平面特徵圖，(對一個樣本，卷積的輸出是M個特徵圖)，對於一個卷積核，要在一個樣本的每一個平面位置進行內積運算，然後沿著H和W方向進行滑動，例如，圖2c表示一個卷積核在輸入數據的一個樣本中右下角的位置進行內積運算的對應圖；圖2d表示卷積的位置向左滑動一格和圖2e表示卷積的位置向上滑動一格。 The input data is shown in Figure 2a (N samples, each sample has C channels, the height of the feature map of each channel is H, and the width is W), and the weight, that is, the convolution kernel is shown in Figure 2b (there are M Convolution kernel, each convolution kernel has C channels, and the height and width are KH and KW respectively). For N samples of input data, the rules of convolution operation are the same. The following explains the process of convolution operation on a sample. On a sample, each of the M convolution kernels must perform the same Operation, each convolution kernel operation obtains a plane feature map, M convolution kernels finally calculate M plane feature maps, (for a sample, the convolution output is M feature maps), for a convolution kernel , it is necessary to perform an inner product operation at each plane position of a sample, and then slide along the H and W directions. For example, Figure 2c shows that a convolution kernel performs an inner product operation at the position of the lower right corner of a sample of input data Corresponding figure; Figure 2d shows that the position of the convolution slides one grid to the left and Figure 2e shows that the position of the convolution slides up one grid.
當第一運算為卷積運算，所述輸入數據為卷積輸入數據，所述權值數據為卷積核，第一複雜度=α*C*kW*kW*M*N*W*C*H；其中，α為卷積系數，取值範圍為大於1；C、kW、kW、M為卷積核四個維度的值，N、W、C、H為卷積輸入數據四個維度的值；如所述第一複雜度大於設定閾值，確定該卷積輸入數據以及卷積核是否為浮點數據，如該卷積輸入數據以及卷積核不為浮點數據，將該卷積輸入數據轉換成浮點數據，將卷積核轉換成浮點數據，然後將卷積輸入數據、卷積核以浮點數據類型執行卷積運算。 When the first operation is a convolution operation, the input data is the convolution input data, the weight data is the convolution kernel, and the first complexity=α*C*kW*kW*M*N*W*C* H; among them, α is the convolution coefficient, and the value range is greater than 1; C, kW, kW, and M are the values of the four dimensions of the convolution kernel, and N, W, C, and H are the values of the four dimensions of the convolution input data. value; if the first complexity is greater than the set threshold, determine whether the convolution input data and the convolution kernel are floating-point data, and if the convolution input data and the convolution kernel are not floating-point data, the volume Convert the convolution input data to floating point data, convert the convolution kernel to floating point data, and then perform the convolution operation on the convolution input data and convolution kernel in the floating point data type.
具體的，該卷積處理的方式可以採用如圖3所示的芯片結構處理，主處理電路(也可以稱為主單元)的數據轉換運算電路可以在第一複雜度大於設定閾值時，將權值的部分或全部卷積核中的數據轉換成定點類型的數據，主處理電路的控制電路將權值的部分或全部卷積核中的數據發送到通過橫向數據輸入介面直接與主處理電路相連的那些基礎處理電路(也可以稱為基礎單元)；在一種可選方案中，主處理電路的控制電路將權值中某個卷積核的數據每次發送一個數或者一部分數給某個基礎處理電路；(例如，對於某一個基礎處理電路，第1次發送第3行第1個數，第2次發送第3行數據中的第2個數，第3次發送第3行的第3個數……，或者第1次發送第3行前兩個數，第二次發送第3行第3和第4個數，第三次發送第3行第5和第6個數……；)在一種可選方案中另一種情況是，主處理電路的控制電路將權值中某幾個卷積核的數據每次各發送一個數者一部分數給某個基礎處理電路；(例如，對於某一個基礎處理電路，第1次發送第3,4,5行每行的第1個數，第2次發送第3,4,5行每行的第2個數，第3次發送第3,4,5行每行的第3個數……，或者第1次發送第3,4,5行每行前兩個數，第二次發送第3,4,5行每行第3和第4個數，第三次發送第3,4,5行每行第5和第6個數……；)主處理電路的控制電路把輸入數據按照卷積的位置進行劃分，主處理電路的控制電路將輸入數據中的部分或全部卷積位置中的數據發送到通過竪向數據輸入介面直接與主處理電路相連的那些基礎處理電路；在一種可選方案中，主處理電路的控制電路將輸入數據中某個卷積位置的數據每次發送一個數或者一部分數給某個基礎處理電路；(例如， 對於某一個基礎處理電路，第1次發送第3列第1個數，第2次發送第3列數據中的第2個數，第3次發送第3列的第3個數……，或者第1次發送第3列前兩個數，第二次發送第3列第3和第4個數，第三次發送第3列第5和第6個數……；)在一種可選方案中另一種情況是，主處理電路的控制電路將輸入數據中某幾個卷積位置的數據每次各發送一個數或者一部分數給某個基礎處理電路；(例如，對於某一個基礎處理電路，第1次發送第3,4,5列每列的第1個數，第2次發送第3,4,5列每列的第2個數，第3次發送第3,4,5列每列的第3個數……，或者第1次發送第3,4,5列每列前兩個數，第二次發送第3,4,5列每列第3和第4個數，第三次發送第3,4,5列每列第5和第6個數……；)基礎處理電路接收到權值的數據之後，將該數據通過其橫向的數據輸出介面傳輸給其相連接下一個基礎處理電路；基礎處理電路接收到輸入數據的數據後，將該數據通過其竪向的數據輸出介面傳輸給與其相連接的下一個基礎處理電路；每一個基礎處理電路對接收到的數據進行運算；在一種可選方案中，基礎處理電路每次計算一組或多組兩個數據的乘法，然後將結果累加到寄存器和/或片上緩存上；在一種可選方案中，基礎處理電路每次計算一組或多組兩個向量的內積，然後將結果累加到寄存器和/或片上緩存上；基礎處理電路計算出結果後，可以將結果從數據輸出介面傳輸出去； 在一種可選方案中，該計算結果可以是內積運算的最終結果或中間結果；具體地，如果該基礎處理電路有直接與主處理電路相連接的輸出介面則從該介面傳輸結果，如果沒有，則向著能夠直接向主處理電路輸出的基礎處理電路的方向輸出結果。 Specifically, the convolution processing method can be processed using the chip structure shown in Figure 3, and the data conversion operation circuit of the main processing circuit (also called the main unit) can convert the weight Part or all of the data in the convolution kernel of the value is converted into fixed-point data, and the control circuit of the main processing circuit sends the data in the part or all of the convolution kernel of the weight to the main processing circuit directly through the horizontal data input interface. Those basic processing circuits (also called basic units); in an optional solution, the control circuit of the main processing circuit sends a number or a part of the data of a certain convolution kernel in the weight to a certain basic Processing circuit; (for example, for a certain basic processing circuit, the first number of the third line is sent for the first time, the second number in the third line of data is sent for the second time, and the third number of the third line is sent for the third time The number..., or send the first two numbers in the third line for the first time, send the third and fourth numbers in the third line for the second time, and send the fifth and sixth numbers in the third line for the third time...; ) Another situation in an optional solution is that the control circuit of the main processing circuit sends the data of some convolution kernels in the weight to a certain basic processing circuit each time; (for example, for For a certain basic processing circuit, the first number of each line of the 3rd, 4th, and 5th lines is sent for the first time, the second number of each line of the 3rd, 4th, and 5th lines is sent for the second time, and the third number is sent for the third time , the third number in each line of line 4, 5..., or the first two numbers in each line of line 3, 4, and 5 are sent for the first time, and the third and third numbers of each line of line 3, 4, and 5 are sent for the second time The 4th number, send the 3rd, 4th, 5th line for the third time, the 5th and 6th number in each line...;) The control circuit of the main processing circuit divides the input data according to the convolution position, and the main processing circuit The control circuit sends the data in some or all of the convolution positions in the input data to those basic processing circuits directly connected to the main processing circuit through the vertical data input interface; in an optional solution, the control circuit of the main processing circuit will The data of a certain convolution position in the input data sends a number or a part of the number to a certain basic processing circuit each time; (for example, For a certain basic processing circuit, the first number in the third column is sent for the first time, the second number in the third column of data is sent for the second time, and the third number in the third column is sent for the third time..., or Send the first two numbers in column 3 for the first time, send the 3rd and 4th numbers in column 3 for the second time, and send the 5th and 6th numbers in column 3 for the third time... ;) In an optional scheme In another case, the control circuit of the main processing circuit sends a number or a part of the data of certain convolution positions in the input data to a certain basic processing circuit at a time; (for example, for a certain basic processing circuit, The first time to send the first number of each column in the 3rd, 4th and 5th columns, the second time to send the second number in each column of the 3rd, 4th and 5th columns, and the third time to send each of the 3rd, 4th and 5th columns The third number in the column..., or the first two numbers in each column of the 3rd, 4th, and 5th columns are sent for the first time, and the 3rd and 4th numbers in each column of the 3rd, 4th, and 5th columns are sent for the second time. Send the 5th and 6th numbers in each column of the 3rd, 4th, and 5th columns for three times...;) After the basic processing circuit receives the data of the weight value, it transmits the data to the connected next through its horizontal data output interface A basic processing circuit; after the basic processing circuit receives the data of the input data, it transmits the data to the next basic processing circuit connected to it through its vertical data output interface; each basic processing circuit processes the received data operation; in one alternative, the basic processing circuit calculates the multiplication of one or more sets of two data each time, and then accumulates the result to the register and/or on-chip cache; in one optional solution, the basic processing circuit Calculate the inner product of one or more sets of two vectors at a time, and then accumulate the result to the register and/or on-chip cache; after the basic processing circuit calculates the result, it can transmit the result from the data output interface; In an optional solution, the calculation result may be the final or intermediate result of the inner product operation; specifically, if the basic processing circuit has an output interface directly connected to the main processing circuit, the result is transmitted from the interface, if there is no , the result is output in the direction of the basic processing circuit that can directly output to the main processing circuit.
基礎處理電路接收到來自其他基礎處理電路的計算結果之後，將該數據傳輸給與其相連接的其他基礎處理電路或者主處理電路；向著能夠直接向主處理電路輸出的方向輸出結果(例如，最下面一行基礎處理電路將其輸出結果直接輸出給主處理電路，其他基礎處理電路從竪向的輸出介面向下傳輸運算結果)；主處理電路接收到各個基礎處理電路內積運算的結果，即可得到輸出結果。 After the basic processing circuit receives the calculation results from other basic processing circuits, it transmits the data to other basic processing circuits or the main processing circuit connected to it; the result is output in a direction that can be directly output to the main processing circuit (for example, the bottom A row of basic processing circuits directly output their output results to the main processing circuit, and other basic processing circuits transmit the calculation results downward from the vertical output interface); the main processing circuit receives the results of the inner product operation of each basic processing circuit, and can obtain Output the result.
參閱圖4a，圖4a為一種矩陣乘以矩陣的運算，如所述第一運算為：矩陣乘矩陣運算，所述輸入數據為所述矩陣乘矩陣運算的第一矩陣，所述權值為所述矩陣乘矩陣運算的第二矩陣；第一複雜度=β*F*G*E*F；其中，β為矩陣系數，取值範圍為大於等於1，F、G為第一矩陣的行、列值，E、F為第二矩陣的行、列值；如所述第一複雜度大於設定閾值，確定該第一矩陣以及第二矩陣是否為浮點數據，如該第一矩陣以及第二矩陣不為浮點數據，將該第一矩陣轉換成浮點數據，將第二矩陣轉換成浮點數據，然後將第一矩陣、第二矩陣以浮點數據類型執行矩陣乘矩陣運算。 Referring to Fig. 4a, Fig. 4a is a kind of operation of multiplying matrix by matrix, as described first operation is: matrix by matrix operation, described input data is the first matrix of described matrix by matrix operation, and described weight is The second matrix of the matrix multiplication matrix operation; the first complexity=β*F*G*E*F; wherein, β is a matrix coefficient, and the value range is greater than or equal to 1, and F and G are the rows of the first matrix, Column value, E, F are row, column value of the second matrix; If the first complexity is greater than the set threshold, determine whether the first matrix and the second matrix are floating-point data, such as the first matrix and the second matrix The second matrix is not floating-point data, convert the first matrix to floating-point data, convert the second matrix to floating-point data, and then perform matrix multiplication of the first matrix and the second matrix in floating-point data type Matrix Operations.
參閱圖4b，使用如圖3所示的裝置完成矩陣乘矩陣的運算； 下面描述計算尺寸是M行L列的矩陣S和尺寸是L行N列的矩陣P的乘法的運算，(矩陣S中的每一行與矩陣P的每一列長度相同，如圖2d所示)所述神經網絡計算裝置擁有K個基礎處理電路：步驟S401b、主處理電路在如第一複雜度大於設定閾值時，將矩陣S和矩陣P轉換成定點類型數據，主處理電路的控制電路將矩陣S中的每一行數據分發到K個基礎處理電路中的某一個上，基礎處理電路將接收到的數據保存在片上緩存和/或寄存器中；具體的，可以發送至K個基礎處理電路中與主處理電路連接的基礎處理電路。 Referring to Fig. 4b, use the device as shown in Fig. 3 to complete the operation of matrix multiplication matrix; The following describes the operation of calculating the multiplication of a matrix S with M rows and L columns and a matrix P with L rows and N columns (each row in the matrix S has the same length as each column of the matrix P, as shown in Figure 2d) The above neural network computing device has K basic processing circuits: step S401b, when the main processing circuit converts the matrix S and matrix P into fixed-point type data when the first complexity is greater than the set threshold, the control circuit of the main processing circuit converts the matrix S Each line of data in is distributed to one of the K basic processing circuits, and the basic processing circuit saves the received data in the on-chip cache and/or register; specifically, it can be sent to the K basic processing circuits to communicate with the main The underlying processing circuit to which the processing circuit connects.
在一種可選方案中，如果S的行數M<=K則，主處理電路的控制電路給M個基礎處理電路分別分發S矩陣的一行；在一種可選方案中，如果S的行數M>K，主處理電路的控制電路給每個基礎處理電路分別分發S矩陣中一行或多行的數據。 In an optional solution, if the number of rows of S M<=K, the control circuit of the main processing circuit distributes a row of the S matrix to M basic processing circuits respectively; in an optional solution, if the number of rows of S is M >K, the control circuit of the main processing circuit distributes data of one or more rows in the S matrix to each basic processing circuit.
S中有Mi行分發到第i個基礎處理電路，這Mi行的集合稱為Ai，如圖2e表示第i個基礎處理電路上將要執行的計算。 There are Mi rows in S distributed to the i-th basic processing circuit, and the set of Mi rows is called Ai, as shown in Figure 2e, which shows the computation to be performed on the i-th basic processing circuit.
在一種可選方案中，在每個基礎處理電路中，例如第i個基礎處理電路中：接收的由主處理電路分發的矩陣Ai，將矩陣Ai保存在第i個基礎處理電路寄存器和/或片上緩存中；優點是減少了之後的數據傳輸量，提高了計算效率，降低了功耗。 In an optional solution, in each basic processing circuit, for example, in the i-th basic processing circuit: receiving the matrix Ai distributed by the main processing circuit, storing the matrix Ai in the i-th basic processing circuit register and/or In the on-chip cache; the advantage is that it reduces the amount of subsequent data transmission, improves computing efficiency, and reduces power consumption.
步驟S402b、主處理電路的控制電路將矩陣P中各部分以廣播的方式傳輸給各個基礎處理電路； 在一種可選方案中，可以將矩陣P中各部分只廣播一次到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對這一次得到的矩陣P的數據進行充分地復用，完成對應與矩陣Ai中每一行的內積運算；本實施例中的復用具體可以為基礎處理電路在計算中重複使用，例如矩陣P的數據的復用，可以是對矩陣P的數據在多次使用。 Step S402b, the control circuit of the main processing circuit transmits each part in the matrix P to each basic processing circuit in a broadcast manner; In an alternative solution, each part of the matrix P can be broadcast only once to the registers or on-chip caches of each basic processing circuit, and the i-th basic processing circuit fully multiplexes the data of the matrix P obtained this time, Complete the inner product operation corresponding to each row in the matrix Ai; the multiplexing in this embodiment can specifically be the repeated use of the basic processing circuit in calculations, for example, the multiplexing of the data of the matrix P can be the data of the matrix P in multiple times used.
在一種可選方案中，主處理電路的控制電路可以將矩陣P中各部分多次廣播到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對每次得到的矩陣P的數據不進行復用，分次完成對應於矩陣Ai中的每一行的內積運算；在一種可選方案中，主處理電路的控制電路可以將矩陣P中各部分多次廣播到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對每次得到的矩陣P的數據進行部分復用，完成對應於矩陣Ai中的每一行的內積運算；在一種可選方案中，每個基礎處理電路，例如第i個基礎處理電路，計算矩陣Ai的數據和矩陣P的數據的內積；步驟S403b、每個基礎處理電路的累加器電路將內積運算的結果進行累加並傳輸回主處理電路。 In an optional solution, the control circuit of the main processing circuit can broadcast each part of the matrix P to the registers or on-chip caches of each basic processing circuit for multiple times, and the i-th basic processing circuit performs a calculation on the data of the matrix P obtained each time. No multiplexing is performed, and the inner product operation corresponding to each row in the matrix Ai is completed in batches; in an alternative solution, the control circuit of the main processing circuit can broadcast each part of the matrix P to each basic processing circuit multiple times. In the register or on-chip cache, the i-th basic processing circuit partially multiplexes the data of the matrix P obtained each time, and completes the inner product operation corresponding to each row in the matrix Ai; in an optional solution, each basic The processing circuit, such as the i-th basic processing circuit, calculates the inner product of the data of the matrix Ai and the data of the matrix P; step S403b, the accumulator circuit of each basic processing circuit accumulates the result of the inner product operation and transmits it back to the main processing circuit.
在一種可選方案中，基礎處理電路可以將每次執行內積運算得到的部分和傳輸回主處理電路進行累加；在一種可選方案中，也可以將每次基礎處理電路執行的內積運算得到的部分和保存在基礎處理電路的寄存器和/或片上緩存中，累加結束之後傳輸回主處理電路； 在一種可選方案中，也可以將每次基礎處理電路執行的內積運算得到的部分和在部分情況下保存在基礎處理電路的寄存器和/或片上緩存中進行累加，部分情況下傳輸到主處理電路進行累加，累加結束之後傳輸回主處理電路。 In an optional solution, the basic processing circuit can transfer the partial sum obtained by each execution of the inner product operation back to the main processing circuit for accumulation; in an optional solution, the inner product operation performed by the basic processing circuit can also be The obtained partial sum is stored in the register and/or on-chip cache of the basic processing circuit, and is transferred back to the main processing circuit after the accumulation is completed; In an optional solution, the sum obtained from the inner product operation performed by the basic processing circuit each time may also be stored in the register and/or on-chip cache of the basic processing circuit for accumulation in some cases, and transferred to the main The processing circuit performs accumulation, and after the accumulation is completed, it is transmitted back to the main processing circuit.
參閱圖4c，為一種矩陣乘以向量的運算示意圖。如所述第一運算為：矩陣乘向量運算，所述輸入數據為所述矩陣乘向量運算的第一矩陣，所述權值為所述矩陣乘向量運算的向量；第一複雜度=β*F*G*F；其中，β為矩陣系數，取值範圍為大於等於1，F、G為第一矩陣的行、列值，F為向量的列值；如所述第一複雜度大於設定閾值，確定該第一矩陣以及向量是否為浮點數據，如該第一矩陣以及向量不為浮點數據，將該第一矩陣轉換成浮點數據，將向量轉換成浮點數據，然後將第一矩陣、向量以浮點數據類型執行矩陣乘向量運算。 Referring to FIG. 4c, it is a schematic diagram of an operation of multiplying a matrix by a vector. For example, the first operation is: matrix multiplication vector operation, the input data is the first matrix of the matrix multiplication vector operation, and the weight is the vector of the matrix multiplication vector operation; first complexity=β* F*G*F; Among them, β is a matrix coefficient, the value range is greater than or equal to 1, F and G are the row and column values of the first matrix, and F is the column value of the vector; if the first complexity is greater than the set Threshold, determine whether the first matrix and vector are floating-point data, if the first matrix and vector are not floating-point data, convert the first matrix to floating-point data, and convert the vector to floating-point data , and then perform matrix multiplication-vector operation on the first matrix and vector in floating-point data type.
參閱圖4d，圖4d提供了了一種矩陣乘向量的實現方法，具體可以包括：步驟S401、主處理電路的數據轉換運算電路將矩陣S中的每一行數據轉換成定點類型的數據，主處理電路的控制電路分發到K個基礎處理電路中的某一個上，基礎處理電路將接收到的分發數據保存在基礎處理電路的片上緩存和/或寄存器中；在一種可選方案中，如果矩陣S的行數M<=K則，主處理電路的控制電路給K個基礎處理電路分別分發S矩陣的一行； 在一種可選方案中，如果矩陣S的行數M>K，則主處理電路的控制電路給每個基礎處理電路分別分發S矩陣中一行或多行的數據。 Referring to Fig. 4d, Fig. 4d provides a kind of implementation method of matrix multiplication vector, specifically can comprise: Step S401, the data conversion operation circuit of main processing circuit converts each row of data in the matrix S into fixed-point type data, main processing circuit The control circuit is distributed to one of the K basic processing circuits, and the basic processing circuit saves the received distributed data in the on-chip cache and/or register of the basic processing circuit; in an optional solution, if the matrix S If the number of rows M<=K, the control circuit of the main processing circuit distributes one row of the S matrix to the K basic processing circuits respectively; In an optional solution, if the number of rows of the matrix S is M>K, the control circuit of the main processing circuit distributes the data of one or more rows in the matrix S to each basic processing circuit.
分發到第i個基礎處理電路的S中的行的集合為Ai，共有Mi個行，如圖2c表示第i個基礎處理電路上將要執行的計算。 The set of rows in S distributed to the i-th basic processing circuit is Ai, and there are a total of Mi rows. Figure 2c shows the calculation to be performed on the i-th basic processing circuit.
在一種可選方案中，在每個基礎處理電路中，例如第i個基礎處理電路中，可以將接收到的分發數據例如矩陣Ai保存在第i個基礎處理電路的寄存器和/或片上緩存中；優點是減少了之後的分發數據的數據傳輸量，提高了計算效率，降低了功耗。 In an optional solution, in each basic processing circuit, for example, in the i-th basic processing circuit, the received distribution data such as the matrix Ai may be stored in the register and/or on-chip cache of the i-th basic processing circuit ; The advantage is to reduce the amount of data transmission of the subsequent distribution data, improve the calculation efficiency, and reduce the power consumption.
步驟S402、主處理電路的數據類型運算電路將向量P轉換成定點類型的數據，主處理電路的控制電路將定點類型的向量P中各部分以廣播的方式傳輸給K個基礎處理電路；在一種可選方案中，主處理電路的控制電路可以將向量P中各部分只廣播一次到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對這一次得到的向量P的數據進行充分地復用，完成對應與矩陣Ai中每一行的內積運算。優點是，減少從主處理電路到基礎處理電路的向量P的重複傳輸的數據傳輸量，提高執行效率，降低傳輸功耗。 Step S402, the data type operation circuit of the main processing circuit converts the vector P into fixed-point data, and the control circuit of the main processing circuit transmits each part of the fixed-point vector P to K basic processing circuits in a broadcast mode; In an optional solution, the control circuit of the main processing circuit can broadcast each part of the vector P to the register or on-chip cache of each basic processing circuit only once, and the ith basic processing circuit fully performs the data of the vector P obtained this time. Multiplexed to complete the inner product operation corresponding to each row in the matrix Ai. The advantage is that it reduces the data transmission amount of repeated transmission of the vector P from the main processing circuit to the basic processing circuit, improves execution efficiency, and reduces transmission power consumption.
在一種可選方案中，主處理電路的控制電路可以將向量P中各部分多次廣播到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對每次得到的向量P的數據不進行復用，分次完成對應於矩陣Ai中的每一行的內積運算；優點是，減少基礎處理電路內部的單次傳輸的向量P的數據傳輸量，並可以降低基礎處理電路緩存和/或寄存器的容量，提高執行效率，降低傳輸功耗，降低成本。 In an optional solution, the control circuit of the main processing circuit can broadcast each part of the vector P to the registers or on-chip caches of each basic processing circuit for multiple times, and the i-th basic processing circuit can obtain the data of the vector P each time Without multiplexing, the inner product operation corresponding to each row in the matrix Ai is completed in batches; the advantage is that the data transmission amount of the single-transmission vector P inside the basic processing circuit is reduced, and the basic processing circuit cache and/or Or the capacity of registers, improve execution efficiency, reduce transmission power consumption, and reduce costs.
在一種可選方案中，主處理電路的控制電路可以將向量P中各部分多次廣播到各個基礎處理電路的寄存器或者片上緩存中，第i個基礎處理電路對每次得到的向量P的數據進行部分復用，完成對應於矩陣Ai中的每一行的內積運算；優點是，減少從主處理電路到基礎處理電路的數據傳輸量，也減少基礎處理電路內部的數據傳輸量，提高執行效率，降低傳輸功耗。 In an optional solution, the control circuit of the main processing circuit can broadcast each part of the vector P to the registers or on-chip caches of each basic processing circuit for multiple times, and the i-th basic processing circuit can obtain the data of the vector P each time Perform partial multiplexing to complete the inner product operation corresponding to each row in the matrix Ai; the advantage is that it reduces the amount of data transmission from the main processing circuit to the basic processing circuit, and also reduces the amount of data transmission inside the basic processing circuit, improving execution efficiency , to reduce transmission power consumption.
步驟S403、K個基礎處理電路的內積運算器電路計算矩陣S和向量P的數據的內積，例如第i個基礎處理電路，計算矩陣Ai的數據和向量P的數據的內積；步驟S404、K個基礎處理電路的累加器電路將內積運算的結果進行累加得到累加結果，將累加結果以定點類型形式傳輸回主處理電路。
Step S403, the inner product operator circuit of K basic processing circuits calculates the inner product of the data of matrix S and vector P, such as the i-th basic processing circuit, calculates the inner product of the data of matrix Ai and the data of vector P;
在一種可選方案中，可以將每次基礎處理電路執行內積運算得到的部分和(部分和即累加結果的一部分，例如累加結果為：F1*G1+F2*G2+F3*G3+F4*G4+F5*G5，那麼部分和可以為：F1*G1+F2*G2+F3*G3的值)傳輸回主處理電路進行累加；優點是，減少了基礎處理電路內部的運算量，提高基礎處理電路的運算效率。 In an optional solution, the partial sum (partial sum is a part of the accumulation result obtained by performing the inner product operation of the basic processing circuit each time, for example, the accumulation result is: F1*G1+F2*G2+F3*G3+F4* G4+F5*G5, then the partial sum can be: the value of F1*G1+F2*G2+F3*G3) is transmitted back to the main processing circuit for accumulation; the advantage is that it reduces the amount of computation inside the basic processing circuit and improves the basic processing The operational efficiency of the circuit.
在一種可選方案中，也可以將每次基礎處理電路執行的內積運算得到的部分和保存在基礎處理電路的寄存器和/或片上緩存中，累加結束之後傳輸回主處理電路；優點是，減少了基礎處理電路和主處理電路之間的數據傳輸量，提高了運算效率，降低了數據傳輸功耗。 In an optional solution, the partial sum obtained by the inner product operation performed by the basic processing circuit each time can also be stored in the register and/or on-chip cache of the basic processing circuit, and transferred back to the main processing circuit after the accumulation is completed; the advantage is that The amount of data transmission between the basic processing circuit and the main processing circuit is reduced, the operation efficiency is improved, and the power consumption of data transmission is reduced.
在一種可選方案中，也可以將每次基礎處理電路執行的內積運算得到的部分和在部分情況下保存在基礎處理電路的寄存器和/或片上緩存中進行累加，部分情況下傳輸到主處理電路進行累加，累加結束之後傳輸回主處理 電路；優點是，減少了基礎處理電路和主處理電路之間的數據傳輸量，提高了運算效率，降低了數據傳輸功耗，減少了基礎處理電路內部的運算量，提高基礎處理電路的運算效率。 In an optional solution, the sum obtained from the inner product operation performed by the basic processing circuit each time may also be stored in the register and/or on-chip cache of the basic processing circuit for accumulation in some cases, and transferred to the main The processing circuit accumulates, and after the accumulation is completed, it is transferred back to the main processing circuit; the advantage is that the amount of data transmission between the basic processing circuit and the main processing circuit is reduced, the calculation efficiency is improved, the power consumption of data transmission is reduced, the calculation amount inside the basic processing circuit is reduced, and the calculation efficiency of the basic processing circuit is improved .
本披露還提供一種集成電路芯片裝置，所述集成電路芯片裝置用於執行神經網絡的正向運算，所述神經網絡包含多層，所述裝置包括：處理電路以及外部介面；所述外部介面，用於接收第一計算指令；所述處理電路，用於解析第一計算指令得到所述第一計算指令在所述正向運算的第i層包含的第一運算、第一計算指令對應的輸入數據以及權值數據；上述i的取值可以為1，如為1時，其輸入數據可以為原始輸入數據，當i大於等於2時，該輸入數據可以為上一層的輸出數據，例如i-1層的輸出數據。 The present disclosure also provides an integrated circuit chip device, the integrated circuit chip device is used to execute the forward operation of the neural network, the neural network includes multiple layers, and the device includes: a processing circuit and an external interface; the external interface is used for For receiving a first calculation instruction; the processing circuit is configured to analyze the first calculation instruction to obtain the first operation contained in the i-th layer of the forward operation by the first calculation instruction and the input data corresponding to the first calculation instruction And weight data; the value of the above i can be 1, if it is 1, its input data can be the original input data, when i is greater than or equal to 2, the input data can be the output data of the previous layer, for example i-1 Layer output data.
所述處理電路，還用於依據該輸入數據、權值數據以及第一運算確定第一運算的第一複雜度，依據所述第一複雜度確定該輸入數據以及權值數據在執行第一運算時的第一數據類型，所述第一數據類型包括：浮點類型或定點類型；所述處理電路，還用於將輸入數據以及權值數據以第一數據類型執行所述正向運算的第i層包含的第一運算。 The processing circuit is further configured to determine the first complexity of the first operation according to the input data, the weight data and the first operation, and determine the input data and the weight data according to the first complexity when performing the first operation When the first data type is used, the first data type includes: floating-point type or fixed-point type; the processing circuit is also used to perform the first forward operation of the input data and weight data in the first data type The first operation contained in layer i.
本披露還揭露了一個神經網絡運算裝置，其包括一個或多個在如圖3所示的芯片，用於從其他處理裝置中獲取待運算數據和控制信息，執行指定的神經網絡運算，執行結果通過I/O介面傳遞給外圍設備。外圍設備譬如攝像頭、顯示器、鼠標、鍵盤、網卡、wifi介面、伺服器。當包含一個以上如圖3 所示的芯片時，如圖3所示的芯片間可以通過特定的結構進行鏈接並傳輸數據，譬如，通過PCIE總線進行互聯並傳輸數據，以支持更大規模的神經網絡的運算。此時，可以共享同一控制系統，也可以有各自獨立的控制系統；可以共享內存，也可以每個加速器有各自的內存。此外，其互聯方式可以是任意互聯拓撲。 This disclosure also discloses a neural network computing device, which includes one or more chips as shown in Figure 3, used to obtain data to be processed and control information from other processing devices, perform specified neural network operations, and execute results It is passed to the peripheral device through the I/O interface. Peripheral equipment such as camera, monitor, mouse, keyboard, network card, wifi interface, server. When including more than one as shown in Figure 3 As shown in the chip, the chips shown in Figure 3 can be linked and transmit data through a specific structure, for example, interconnect and transmit data through a PCIE bus to support larger-scale neural network operations. At this time, the same control system can be shared, or there can be independent control systems; the memory can be shared, or each accelerator can have its own memory. In addition, its interconnection method can be any interconnection topology.
該神經網絡運算裝置具有較高的兼容性，可通過PCIE介面與各種類型的伺服器相連接。 The neural network computing device has high compatibility and can be connected with various types of servers through the PCIE interface.
本披露還揭露了一個組合處理裝置，其包括上述的神經網絡運算裝置，通用互聯介面，和其他處理裝置(即通用處理裝置)。神經網絡運算裝置與其他處理裝置進行交互，共同完成用戶指定的操作。如圖5a為組合處理裝置的示意圖。 This disclosure also discloses a combined processing device, which includes the above-mentioned neural network computing device, a general interconnection interface, and other processing devices (ie, a general processing device). The neural network computing device interacts with other processing devices to jointly complete operations specified by the user. Figure 5a is a schematic diagram of the combination processing device.
其他處理裝置，包括中央處理器CPU、圖形處理器GPU、神經網絡處理器等通用/專用處理器中的一種或以上的處理器類型。其他處理裝置所包括的處理器數量不做限制。其他處理裝置作為神經網絡運算裝置與外部數據和控制的介面，包括數據搬運，完成對本神經網絡運算裝置的開啟、停止等基本控制；其他處理裝置也可以和神經網絡運算裝置協作共同完成運算任務。 Other processing devices include one or more types of general-purpose/special-purpose processors such as central processing unit CPU, graphics processing unit GPU, and neural network processor. The number of processors included in other processing devices is not limited. Other processing devices serve as the interface between the neural network computing device and external data and control, including data transfer, and complete the basic control of starting and stopping the neural network computing device; other processing devices can also cooperate with the neural network computing device to complete computing tasks.
通用互聯介面，用於在所述神經網絡運算裝置與其他處理裝置間傳輸數據和控制指令。該神經網絡運算裝置從其他處理裝置中獲取所需的輸入數據，寫入神經網絡運算裝置片上的存儲裝置；可以從其他處理裝置中獲取控制指令，寫入神經網絡運算裝置片上的控制緩存；也可以讀取神經網絡運算裝置的存儲模塊中的數據並傳輸給其他處理裝置。 The universal interconnection interface is used for transmitting data and control instructions between the neural network computing device and other processing devices. The neural network computing device obtains the required input data from other processing devices and writes it into the storage device on the neural network computing device; it can obtain control instructions from other processing devices and writes it into the control cache on the neural network computing device chip; The data in the storage module of the neural network computing device can be read and transmitted to other processing devices.
如圖5b所示，可選的，該結構還包括存儲裝置，用於保存在本運算單元/運算裝置或其他運算單元所需要的數據，尤其適用於所需要運算的數據在本神經網絡運算裝置或其他處理裝置的內部存儲中無法全部保存的數據。 As shown in Figure 5b, optionally, the structure also includes a storage device, which is used to save the data required by this computing unit/computing device or other computing units, especially for the data required for computing in this neural network computing device or data that cannot be fully stored in the internal storage of other processing devices.
該組合處理裝置可以作為手機、機器人、無人機、視頻監控設備等設備的SOC片上系統，有效降低控制部分的核心面積，提高處理速度，降低整體功耗。此情況時，該組合處理裝置的通用互聯介面與設備的某些部件相連接。某些部件譬如攝像頭、顯示器、鼠標、鍵盤、網卡、wifi介面。 The combined processing device can be used as a SOC system on a mobile phone, robot, drone, video surveillance equipment and other equipment, effectively reducing the core area of the control part, increasing the processing speed, and reducing the overall power consumption. In this case, the common interconnection interface of the combined processing device is connected with certain components of the equipment. Some components such as camera, monitor, mouse, keyboard, network card, wifi interface.
請參照圖5c，圖5c為本披露實施例提供的一種神經網絡處理器板卡的結構示意圖。如圖5c所示，上述神經網絡處理器板卡10包括神經網絡芯片封裝結構11、第一電氣及非電氣連接裝置12和第一基板(substrate)13。
Please refer to FIG. 5c, which is a schematic structural diagram of a neural network processor board provided by an embodiment of the present disclosure. As shown in FIG. 5 c , the neural
本披露對於神經網絡芯片封裝結構11的具體結構不作限定，可選的，如圖5d所示，上述神經網絡芯片封裝結構11包括：神經網絡芯片111、第二電氣及非電氣連接裝置112、第二基板113。
The present disclosure does not limit the specific structure of the neural network
本披露所涉及的神經網絡芯片111的具體形式不作限定，上述的神經網絡芯片111包含但不限於將神經網絡處理器集成的神經網絡晶片，上述晶片可以由硅材料、鍺材料、量子材料或分子材料等製成。根據實際情況(例如：較嚴苛的環境)和不同的應用需求可將上述神經網絡晶片進行封裝，以使神經網絡晶片的大部分被包裹住，而將神經網絡晶片上的引腳通過金線等導體連到封裝結構的外邊，用於和更外層進行電路連接。
The specific form of the
本披露對於神經網絡芯片111的具體結構不作限定，可選的，請參照圖3所示的裝置。
The present disclosure does not limit the specific structure of the
本披露對於第一基板13和第二基板113的類型不做限定，可以是印制電路板(printed circuit board，PCB)或(printed wiring board，PWB)，還可能為其它電路板。對PCB的製作材料也不做限定。
The present disclosure does not limit the types of the
本披露所涉及的第二基板113用於承載上述神經網絡芯片111，通過第二電氣及非電氣連接裝置112將上述的神經網絡芯片111和第二基板113進行連接得到的神經網絡芯片封裝結構11，用於保護神經網絡芯片111，便於將神經網絡芯片封裝結構11與第一基板13進行進一步封裝。
The
對於上述具體的第二電氣及非電氣連接裝置112的封裝方式和封裝方式對應的結構不作限定，可根據實際情況和不同的應用需求選擇合適的封裝方式並進行簡單地改進，例如：倒裝芯片球柵陣列封裝(Flip Chip Ball Grid Array Package，FCBGAP)，薄型四方扁平式封裝(Low-profile Quad Flat Package，LQFP)、帶散熱器的四方扁平封裝(Quad Flat Package with Heat sink，HQFP)、無引腳四方扁平封裝(Quad Flat Non-lead Package，QFN)或小間距四方扁平式封裝(Fine-pitch Ball Grid Package，FBGA)等封裝方式。
There is no limitation on the packaging method and the structure corresponding to the packaging method of the above-mentioned second electrical and non-electrical connecting
倒裝芯片(Flip Chip)，適用於對封裝後的面積要求高或對導線的電感、信號的傳輸時間敏感的情況下。除此之外可以用引線鍵合(Wire Bonding)的封裝方式，減少成本，提高封裝結構的靈活性。 Flip chip is suitable for the case where the area after packaging is high or the inductance of the wire and the transmission time of the signal are sensitive. In addition, wire bonding (Wire Bonding) packaging can be used to reduce costs and improve the flexibility of the packaging structure.
球柵陣列(Ball Grid Array)，能夠提供更多引腳，且引腳的平均導線長度短，具備高速傳遞信號的作用，其中，封裝可以用引腳網格陣列封裝(Pin Grid Array，PGA)、零插拔力(Zero Insertion Force，ZIF)、單邊接觸連接(Single Edge Contact Connection，SECC)、觸點陣列(Land Grid Array，LGA)等來代替。 Ball Grid Array (Ball Grid Array), which can provide more pins, and the average wire length of the pins is short, and has the function of high-speed signal transmission. Among them, the package can be packaged with Pin Grid Array (PGA) , Zero Insertion Force (ZIF), single edge contact connection (Single Edge Contact Connection, SECC), contact array (Land Grid Array, LGA) etc. instead.
可選的，採用倒裝芯片球柵陣列(Flip Chip Ball Grid Array)的封裝方式對神經網絡芯片111和第二基板113進行封裝，具體的神經網絡芯片封裝結構的示意圖可參照圖6。如圖6所示，上述神經網絡芯片封裝結構包括：神經網絡芯片21、焊盤22、焊球23、第二基板24、第二基板24上的連接點25、引腳26。
Optionally, the
其中，焊盤22與神經網絡芯片21相連，通過在焊盤22和第二基板24上的連接點25之間焊接形成焊球23，將神經網絡芯片21和第二基板24連接，即實現了神經網絡芯片21的封裝。
Wherein, the
引腳26用於與封裝結構的外部電路(例如，神經網絡處理器板卡10上的第一基板13)相連，可實現外部數據和內部數據的傳輸，便於神經網絡芯片21或神經網絡芯片21對應的神經網絡處理器對數據進行處理。對於引腳的類型和數量本披露也不作限定，根據不同的封裝技術可選用不同的引腳形式，並遵從一定規則進行排列。
可選的，上述神經網絡芯片封裝結構還包括絕緣填充物，置於焊盤22、焊球23和連接點25之間的空隙中，用於防止焊球與焊球之間產生干擾。
Optionally, the above neural network chip packaging structure further includes insulating fillers placed in the gaps between the
其中，絕緣填充物的材料可以是氮化硅、氧化硅或氧氮化硅；干擾包含電磁干擾、電感干擾等。 Wherein, the material of the insulating filler may be silicon nitride, silicon oxide or silicon oxynitride; interference includes electromagnetic interference, inductive interference, and the like.
可選的，上述神經網絡芯片封裝結構還包括散熱裝置，用於散發神經網絡芯片21運行時的熱量。其中，散熱裝置可以是一塊導熱性良好的金屬片、散熱片或散熱器，例如，風扇。
Optionally, the above neural network chip packaging structure further includes a heat dissipation device for dissipating the heat generated by the
舉例來說，如圖6a所示，神經網絡芯片封裝結構11包括：神經網絡芯片21、焊盤22、焊球23、第二基板24、第二基板24上的連接點25、引腳26、絕緣填充物27、散熱膏28和金屬外殼散熱片29。其中，散熱膏28和金屬外殼散熱片29用於散發神經網絡芯片21運行時的熱量。
For example, as shown in FIG. 6a, the neural network
可選的，上述神經網絡芯片封裝結構11還包括補強結構，與焊盤22連接，且內埋於焊球23中，以增強焊球23與焊盤22之間的連接強度。
Optionally, the neural network
其中，補強結構可以是金屬線結構或柱狀結構，在此不做限定。 Wherein, the reinforcement structure may be a metal wire structure or a column structure, which is not limited here.
本披露對於第一電氣及非電氣裝置12的具體形式也不作限定，可參照第二電氣及非電氣裝置112的描述，即通過焊接的方式將神經網絡芯片封裝結構11進行封裝，也可以採用連接線連接或插拔方式連接第二基板113和第一基板13的方式，便於後續更換第一基板13或神經網絡芯片封裝結構11。
This disclosure does not limit the specific form of the first electrical and
可選的，第一基板13包括用於擴展存儲容量的內存單元的介面等，例如：同步動態隨機存儲器(Synchronous Dynamic Random Access Memory，SDRAM)、雙倍速率同步動態隨機存儲器(Double Date Rate SDRAM，DDR)等，通過擴展內存提高了神經網絡處理器的處理能力。
Optionally, the
第一基板13上還可包括快速外部設備互連總線(Peripheral Component Interconnect-Express，PCI-E或PCIe)介面、小封裝可熱插拔(Small Form-factor Pluggable，SFP)介面、以太網介面、控制器局域網總線(Controller Area Network，CAN)介面等等，用於封裝結構和外部電路之間的數據傳輸，可提高運算速度和操作的便利性。
The
將神經網絡處理器封裝為神經網絡芯片111，將神經網絡芯片111封裝為神經網絡芯片封裝結構11，將神經網絡芯片封裝結構11封裝為神經
網絡處理器板卡10，通過板卡上的介面(插槽或插芯)與外部電路(例如：計算機主板)進行數據交互，即直接通過使用神經網絡處理器板卡10實現神經網絡處理器的功能，並保護神經網絡芯片111。且神經網絡處理器板卡10上還可添加其他模塊，提高了神經網絡處理器的應用範圍和運算效率。
The neural network processor is packaged as a
在一個實施例裏，本公開公開了一個電子裝置，其包括了上述神經網絡處理器板卡10或神經網絡芯片封裝結構11。
In one embodiment, the present disclosure discloses an electronic device, which includes the neural
電子裝置包括數據處理裝置、機器人、電腦、打印機、掃描儀、平板電腦、智能終端、手機、行車記錄儀、導航儀、傳感器、攝像頭、伺服器、相機、攝像機、投影儀、手錶、耳機、移動存儲、可穿戴設備、交通工具、家用電器、和/或醫療設備。 Electronic devices include data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, mobile phones, driving recorders, navigators, sensors, cameras, servers, cameras, video cameras, projectors, watches, earphones, mobile storage, wearables, vehicles, home appliances, and/or medical devices.
所述交通工具包括飛機、輪船和/或車輛；所述家用電器包括電視、空調、微波爐、冰箱、電飯煲、加濕器、洗衣機、電燈、燃氣灶、油煙機；所述醫療設備包括核磁共振儀、B型超音波掃描儀和/或心電圖儀。 Said vehicles include airplanes, ships and/or vehicles; said household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, range hoods; said medical equipment includes nuclear magnetic resonance machine, B-mode ultrasound scanner and/or electrocardiograph.
以上所述的具體實施例，對本披露的目的、技術方案和有益效果進行了進一步詳細說明，所應理解的是，以上所述僅為本披露的具體實施例而已，並不用於限制本披露，凡在本披露的精神和原則之內，所做的任何修改、等同替換、改進等，均應包含在本披露的保護範圍之內。 The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the disclosure in detail. It should be understood that the above descriptions are only specific embodiments of the disclosure and are not intended to limit the disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.
Claims (16)
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| ??201711469857.8 | 2017-12-28 | ||
| CN201711469857.8 | 2017-12-28 | ||
| CN201711469857.8A CN109978131B (en) | 2017-12-28 | 2017-12-28 | Integrated circuit chip apparatus, method and related product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201931215A TW201931215A (en) | 2019-08-01 |
| TWI791725B true TWI791725B (en) | 2023-02-11 |
Family
ID=67075536
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW107147412A TWI791725B (en) | 2017-12-28 | 2018-12-27 | Neural network operation method, integrated circuit chip device and related products |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109978131B (en) |
| TW (1) | TWI791725B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109993284B (en) * | 2017-12-29 | 2020-08-04 | 中科寒武纪科技股份有限公司 | Integrated circuit chip device and related product |
| CN110458285B (en) * | 2019-08-14 | 2021-05-14 | 中科寒武纪科技股份有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| TWI733334B (en) * | 2020-02-15 | 2021-07-11 | 財團法人工業技術研究院 | Convolutional neural-network calculating apparatus and operation methods thereof |
| CN112491555B (en) * | 2020-11-20 | 2022-04-05 | 山西智杰软件工程有限公司 | Medical electronic signature processing method and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106126481A (en) * | 2016-06-29 | 2016-11-16 | 华为技术有限公司 | A kind of computing engines and electronic equipment |
| CN107229967A (en) * | 2016-08-22 | 2017-10-03 | 北京深鉴智能科技有限公司 | A kind of hardware accelerator and method that rarefaction GRU neutral nets are realized based on FPGA |
| EP3242253A1 (en) * | 2016-05-03 | 2017-11-08 | Imagination Technologies Limited | Hardware implementation of a convolutional neural network |
| US20170357891A1 (en) * | 2016-05-26 | 2017-12-14 | The Governing Council Of The University Of Toronto | Accelerator for deep neural networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017181419A1 (en) * | 2016-04-22 | 2017-10-26 | 北京中科寒武纪科技有限公司 | Submatrix operation device and method |
-
2017
- 2017-12-28 CN CN201711469857.8A patent/CN109978131B/en active Active
-
2018
- 2018-12-27 TW TW107147412A patent/TWI791725B/en active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3242253A1 (en) * | 2016-05-03 | 2017-11-08 | Imagination Technologies Limited | Hardware implementation of a convolutional neural network |
| US20170357891A1 (en) * | 2016-05-26 | 2017-12-14 | The Governing Council Of The University Of Toronto | Accelerator for deep neural networks |
| CN106126481A (en) * | 2016-06-29 | 2016-11-16 | 华为技术有限公司 | A kind of computing engines and electronic equipment |
| CN107229967A (en) * | 2016-08-22 | 2017-10-03 | 北京深鉴智能科技有限公司 | A kind of hardware accelerator and method that rarefaction GRU neutral nets are realized based on FPGA |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109978131B (en) | 2020-05-22 |
| TW201931215A (en) | 2019-08-01 |
| CN109978131A (en) | 2019-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI793225B (en) | Method for neural network training and related product | |
| TWI791725B (en) | Neural network operation method, integrated circuit chip device and related products | |
| US11748605B2 (en) | Integrated circuit chip device | |
| TWI768159B (en) | Integrated circuit chip apparatus and related product | |
| US20200257752A1 (en) | Integrated circuit chip apparatus | |
| CN111242294A (en) | Integrated circuit chip device and related product | |
| TWI767098B (en) | Method for neural network forward computation and related product | |
| TWI793224B (en) | Integrated circuit chip apparatus and related product | |
| CN109977446B (en) | Integrated circuit chip device and related product | |
| CN109978157B (en) | Integrated circuit chip device and related product | |
| CN109978148B (en) | Integrated circuit chip device and related product | |
| CN109978156B (en) | Integrated circuit chip device and related product | |
| TWI767097B (en) | Integrated circuit chip apparatus and related product | |
| TWI768160B (en) | Integrated circuit chip apparatus and related product | |
| CN109978158B (en) | Integrated circuit chip device and related product | |
| TWI795482B (en) | Integrated circuit chip apparatus and related product |