CN108229644A - The device of compression/de-compression neural network model, device and method - Google Patents
The device of compression/de-compression neural network model, device and method Download PDFInfo
- Publication number
- CN108229644A CN108229644A CN201611159629.6A CN201611159629A CN108229644A CN 108229644 A CN108229644 A CN 108229644A CN 201611159629 A CN201611159629 A CN 201611159629A CN 108229644 A CN108229644 A CN 108229644A
- Authority
- CN
- China
- Prior art keywords
- neural network
- parameter
- compressed
- compression
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Abstract
A kind of device of compression/de-compression neural network model, device and method.Including step：Obtain the parameter to be compressed of neural network model；The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network parameter of low-dimensional；The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.The present invention realizes the device of compression/de-compression neural network model with autocoding neural network algorithm, can reduce the parameter of neural network model, be conducive to storing and transmitting for model.
Description
Technical field
The present invention relates to neural network model compression/decompression algorithm applied technical fields, relate more specifically to a kind of pressure
The device and equipment of contracting/decompression neural network model further relate to a kind of method of compression/de-compression neural network model.
Background technology
In recent years, neural network algorithm is widely applied to every field, with problem complexity and to accuracy rate requirement
Continuous improvement, neural network model depth is continuously increased, and the thing followed is the explosive growth of number of parameters, this is to nerve
Storing and transmitting for network model brings great inconvenience.Imagine each application on mobile phone in future and have deep learning
Ability, but each application will transmit, in storage G neural network model parameter, this is clearly unreasonable.
Traditional dimension reduction method be mostly it is linear, such as PCA (Principal Component Analysis, it is main into
Analysis) variance the best part direction in high dimensional data is chosen, by selecting these directions, obtain low comprising most information
Dimension table shows.However, the linear property of PCA methods causes the characteristic type extracted to have considerable restraint.
Invention content
In view of this, the purpose of the present invention is to provide a kind of with autocoding neural network algorithm compression/de-compression god
Device and method through network model, to solve above-mentioned at least one technical problem.
According to an aspect of the present invention, a kind of method of compression/de-compression neural network model is provided, including step：
S1：Obtain the parameter to be compressed of neural network model；
S2：The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network of low-dimensional
Parameter；
S3：The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
Further, step S1 includes：Traversal selection is carried out to the parameter to be compressed of neural network model, until choose
The quantity of parameter to be compressed is equal to the dimension of setting.
Further, step S1 includes：Traversal selection is carried out to the parameter to be compressed of neural network model, waits to press to described
Contracting parameter carries out rarefaction, the parameter to be compressed of selection is judged, the parameter to be compressed less than given threshold is arranged to
0, it chooses the non-zero entry after rarefaction and marks the position coordinates of non-zero entry, set until the quantity for the parameter to be compressed chosen is equal to
Fixed dimension.
Further, the traversal is chosen obtains each layer according to the sequencing of structure neural network model and waits to press successively
Contracting parameter.
Further, step S2 includes sub-step：
S21：Build autocoding neural network based on multilayer perceptron, the input layer of autocoding neural network and
Output layer number of nodes is identical, and the number of hidden nodes is less than input layer number；
S22：Parameter to be compressed is inputted, forward conduction calculating is carried out to every layer of neuron of autocoding neural network, is obtained
To the activation value of each layer；
S23：Output is enabled to be equal to input, the residual error of output layer and each layer neuron is obtained using backward conduction algorithm；
S24：Using gradient descent method update weights W and biasing B, output is made to become closer to input；
S25：After weights and biasing convergence, the neural network parameter of the value, as low-dimensional of hidden layer is exported.
Further, it is unziped it using the subnetwork of autocoding neural network in step S21, is restored to output
In layer.
According to another aspect of the invention, a kind of device of compression/de-compression neural network model, including parameter acquiring mould
Block, model compression module, model memory module and model decompression module, wherein,
Parameter acquisition module, for obtaining the parameter to be compressed of neural network model；
Model compression module for compressing the parameter to be compressed using neural network algorithm, and is trained, and is obtained low
The neural network parameter of dimension；
Model decompression module for decompressing the neural network parameter of low-dimensional, forms the neural network parameter of recovery；With
And
Memory module, for storing the neural network parameter of the parameter to be compressed of neural network model, low-dimensional and recovery
Neural network parameter.
Further, it in the model compression module, compresses the parameter to be compressed and is calculated by autocoding neural network
Method is compressed, and autocoding neural network is divided into compression network, intermediate hidden layer reconciliation compression network, the compression network input
Parameter to be compressed is exported to intermediate hidden layer, and the number of nodes inputted is more than the number of nodes of output.
Further, the autocoding neural network is built based on multilayer perceptron.
Further, in the model decompression module, the neural network parameter for decompressing low-dimensional passes through the decompression net
Network is decompressed, the neural network parameter of the decompression network inputs low-dimensional, restores the quantity of neural network parameter.
In accordance with a further aspect of the present invention, a kind of equipment of compression/de-compression neural network model is provided, including：
Memory, for storing executable instruction；And
Processor, for performing the executable instruction stored in memory, to perform following operation：
Obtain the parameter to be compressed of neural network model；
The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network ginseng of low-dimensional
Number；
The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
Based on above-mentioned technical proposal it is found that apparatus and method of the present invention has the advantages that：
(1) neural network model parameter can be effectively reduced using the device, saves the memory headroom needed for storage model, have
Conducive to the transmission and transplanting of model；
(2) network parameter can be restored to a greater degree in decompression using correlation method, compared to general linear drop
Dimension method has higher accuracy rate.
(3) by the way that there is minimum node in hidden layer, number of parameters can be effectively reduced, save memory, be conducive to model
It stores and transmits；Neural network model is decompressed when in use, while ensures accuracy rate, and neural network algorithm is made preferably to apply
To in practice；
(4) general compression method is compared, and the model of neural network is compressed with neural network algorithm, can realize fortune
The multiplexing of unit is calculated, saves memory.
Description of the drawings
Fig. 1 is to be shown according to the integrally-built of device of the compression/de-compression neural network model of one embodiment of the invention
Example block diagram；
Fig. 2 is a kind of parameter acquiring in the device according to the compression/de-compression neural network model of one embodiment of the invention
The example block diagram of module；
Fig. 3 is a kind of autocoding in the device according to the compression/de-compression neural network model of one embodiment of the invention
The example block diagram of neural network structure；
Fig. 4 is a kind of model compression in the device according to the compression/de-compression neural network model of one embodiment of the invention
The example block diagram of module.
Fig. 5 is model decompression a kind of in the device according to the compression/de-compression neural network model of one embodiment of the invention
The example block diagram of contracting module.
Fig. 6 is the method flow diagram according to the compression/de-compression neural network model of one embodiment of the invention.
Fig. 7 is the block diagram according to the equipment of the compression/de-compression neural network model of one embodiment of the invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in further detail.By described in detail below, other aspects of the invention, advantage and protrusion
Feature will become obvious those skilled in the art.
In the present specification, it is following only to illustrate for describing the various embodiments of the principle of the invention, it should not be with any
Mode is construed as limiting the scope of the invention.Described below with reference to attached drawing is used to help comprehensive understanding by claim and its waits
The exemplary embodiment of the present invention that jljl limits.It is described below to help to understand, but these details including a variety of details
It is considered as being only exemplary.Therefore, it will be appreciated by those of ordinary skill in the art that not departing from the scope of the present invention and essence
In the case of god, embodiment described herein can be made various changes and modifications.In addition, it rises for clarity and brevity
See, the description of known function and structure is omitted.In addition, through attached drawing, same reference numerals are used for identity function and operation.
The embodiment of the present invention provides the device of compression/de-compression neural network model, can be by trained neural network
Model parameter is compressed, and can save the memory space of model, is conducive to neural network being transplanted to small memory device.
Fig. 1 is integrally-built for the device of compression/de-compression neural network model provided according to embodiment of the present invention
Example block diagram.The wherein device of the compression/de-compression neural network model, including parameter acquisition module, model compression module, mould
Type memory module and model decompression module.
Wherein, parameter acquisition module, for obtaining the parameter to be compressed of neural network model；
It is specific can be used for obtaining the parameter to be compressed of neural network module and pre-processed (such as treat pressure
Contracting parameter carries out rarefaction), the input for model compression module is prepared.Wherein, acquisition modes can be to neural network mould
The parameter to be compressed of type carries out traversal selection, until the quantity for the parameter to be compressed chosen is equal to the dimension of setting.
Can treat compression parameters to carry out rarefaction for above-mentioned pretreatment, when need to treat compression parameters carry out it is dilute
During thinization, it can include：Traversal selection is carried out to the parameter to be compressed of neural network model, the parameter to be compressed is carried out dilute
Thinization judges the parameter to be compressed of selection the parameter to be compressed less than given threshold is set to 0, after choosing rarefaction
Non-zero entry and the position coordinates for marking non-zero entry, until the quantity for the parameter to be compressed chosen is equal to the dimension of setting.This is sparse
Change can effectively reduce neural network model parameter, save the memory headroom needed for storage model, be conducive to the transmission and shifting of model
It plants.
For parameter to be compressed, network node, weights, training rate, excitation function and biasing can be included.
Preferably, parameter acquisition module exports the parameter of setting dimension each time, until having traversed neural network model
Whole parameters.
Fig. 2 shows for a kind of parameter acquisition module in the device of the compression/de-compression neural network model of the embodiment of the present invention
It is intended to.The module obtains neural network model one and shares 1 parameter.As shown in Fig. 2, to one in convolutional neural networks frame
The neural network framework built in (caffe, Convolution Architecture For Feature ECtraction),
The parameter stored in Parameter File is W [l], and each input vector of model compression module is X [linput], with Boolean type array
Label [l] marks rarefaction situation.It is w that if certain, which reads parameter,i, threshold value threshold, at this time X [linput] before j-1
Item non-empty, then：If wiAbsolute value be more than or equal to threshold, then be stored in array X [linput] in, and by Label
[i] puts 1, and reads next parameter；If the absolute value for reading parameter is less than threshold value, Label [i] is set to 0 and read next
A parameter；Until array X [linput] pile (namely realizing that the quantity for the parameter to be compressed chosen is equal to the dimension of setting).
Wherein, model compression module for compressing the parameter to be compressed, is trained using neural network algorithm, obtained
Obtain the neural network parameter of low-dimensional.
Specifically, it is compressed by autocoding (Auto-encoder) neural network algorithm, the autocoding god
It is built based on multilayer perceptron (MLP) through network, autocoding neural network is divided into compression (encoder) network, hidden
(coder) layer and decompression (decoder) network, the output node number of the input reconciliation compression network of hidden layer network is identical, hidden
The number of nodes of layer is less than both of the above.The compression network inputs parameter to be compressed, exports to hidden layer, and the number of nodes inputted is big
In the number of nodes of output.Network is decompressed equally using MLP structures, input layer is coder layers, and output layer is inputted with encoder
Node layer number is identical.
Fig. 3 illustrates the structure of autocoding neural network in this example.Wherein compression network (being equivalent to input layer) and
It is all three layers of MLP network to decompress network (being equivalent to output layer), and input layer is all mutually l with output layer number of nodesinput, it is intermediate hidden
The number of nodes of layer (coder layers) is at least lcompress, i.e.,：lcompress＜ linput, connect entirely between layers.It is logical in this way
It crosses the compressed number of nodes of neural network algorithm to reduce, so as to reduce memory space.
Fig. 4 illustrates a kind of process of model compression, the i.e. training process of autocoding neural network.
Weights are initialized after putting up autocoding neural network as shown in Figure 3；
X [the l that parameter extraction is obtainedinput] input as model compression module, each layer is calculated in propagated forward
The result of weight, biasing and output layer；
The value of node layer and X [l will be exportedinput] be compared, calculate residual error；
Weight and biasing are updated with gradient descent method；Stop changing when error is sufficiently small or reaches maximum frequency of training
Generation；Output Y [l coder layers intermediatecompress] it is input parameter X [linput] low-dimensional represent.
It preserves by the structured file of compression neural network model, Boolean type array Label [l], decoder network structures text
Part, decoder network parameters WdAnd coder layers of output Y [lcompress], number necessary to these are decompression neural network models
According to.
For model decompression module, for decompressing the neural network parameter of low-dimensional, the neural network parameter of recovery is formed,
It is placed into neural network；Wherein, it decompresses and is decompressed also by above-described autocoding neural network, include decompression
Contracting network, the neural network parameter of compression network input low-dimensional restore the quantity of neural network parameter, and by the god of recovery
It is placed into network through network parameter is corresponding.
Fig. 5 illustrates a kind of process for decompressing neural network model.By Y [lcompress] parameter is input to as Wd's
Decoder networks obtain length as linputOutput X ' [linput].If the neural network model parameter after decompression is W ' [l],
And pass through and read the value of array Label [l] by X ' [linput] correspond in W ' [l].If Label [i] is 0, illustrate to carry in parameter
The absolute value of W [i] is less than the threshold value of rarefaction when taking, and is omitted, then X ' [linput] in there is no its respective items, W ' [i] value is 0；
If Label [i] is 1, the value of X ' [j] is assigned to W ' [i]；Nerve net after having traversed array Label [l] and being decompressed
Network model parameter is W ' [l].
It is shown in Figure 1 for memory module, for storing the nerve of the parameter to be compressed of neural network model, low-dimensional
Network parameter (namely compressed parameter) and the neural network parameter restored.
Optionally, after parameter acquisition module uses rarefaction, memory module is additionally operable to label during storage rarefaction.
A kind of typicalness overall workflow of above device is as follows：
The intact nervous network model for including parameter and structure to one.It first passes through parameter extraction module and extracts a fixed number
Measure parameter；It compresses to obtain the low of parameter using autocoding (auto-encoder) neural network algorithm in model compression module
Dimension table shows, repeats above procedure and has compressed all parameters；Store corresponding parameter and network structure；By low-dimensional parameter during decompression
As the input of decompression (decoder) network, recovery obtains higher-dimension parameter and correspondence is put back by compression network model, weighs
Multiple above procedure decompresses all parameters；The parameter correspondence that decompression is obtained is put back to by compression network model, completes decompression god
Through network model process.
The device of above-described embodiment is to be applied to the situation that parameter to be compressed carries out rarefaction.Another situation be not into
Row rarefaction in the case of this kind, all may be used for Artificial Neural Network Structures and parameter and autocoding network structure and parameter
With the setting with rarefaction similarly hereinafter, only parameter extraction, model storage and decompression process in terms of different from.It is compressed
Neural network model parameter is W [l], and each input vectors of auto-encoder are X [linput].It is successively read parameter wi, and enable
xi=wi, until i=linput.After completing one group of compression process, continue to read the parameter in W [l].
Due to not needing to the rarefaction situation of flag parameters, data to be saved is：By the knot of compression neural network model
Structure file, decompression network structure file, compression network parameter WdAnd the output Y [l of hidden layercompress]。
Decompression process is first by Y [lcompress] parameter is input to as WdDecompression network, obtain length as linputIt is defeated
Go out X ' [linput].If the neural network model parameter after decompression is W ' [l], it is successively read x 'i, and enable w 'i=x 'i；If i=
Input then decompresses next group of parameter, until W ' [l] is all assigned.
Based on same inventive concept, the embodiment of the present invention also provides a kind of method of compression/de-compression neural network model,
It is shown in Figure 6, including step：
S1：Obtain the parameter to be compressed of neural network model；
S2：The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network of low-dimensional
Parameter；
S3：The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
For step S1, can specifically include：Traversal selection is carried out to the parameter to be compressed of neural network model, until
The quantity for the parameter to be compressed chosen is equal to the dimension of setting.
Optionally, compression parameters can also be treated to be pre-processed, the pretreatment can be treat compression parameters carry out it is dilute
Thinization when needing to treat compression parameters progress rarefaction, can include：Parameter to be compressed progress time to neural network model
Selection is gone through, rarefaction is carried out to the parameter to be compressed, the parameter to be compressed of selection is judged, less than treating for given threshold
Compression parameters are set to 0, and are chosen the non-zero entry after rarefaction and are marked the position coordinates of non-zero entry, until the parameter to be compressed chosen
Quantity be equal to setting dimension.The rarefaction can effectively reduce neural network model parameter, save interior needed for storage model
Space is deposited, is conducive to the transmission and transplanting of model.
When needing to treat compression parameters progress rarefaction, step S1 includes：To the parameter to be compressed of neural network model
Traversal selection is carried out, rarefaction is carried out to the parameter to be compressed, the parameter to be compressed of selection is judged, less than setting threshold
The parameter to be compressed of value is set to 0, and is chosen the non-zero entry after rarefaction and is marked the position coordinates of non-zero entry, until that chooses waits to press
The quantity of contracting parameter is equal to the dimension of setting.And it is corresponding, such as using rarefaction step, then in step S3, needed after decompression
Neural network parameter is placed according to the mark position of non-zero entry.
Carry out rarefaction or during non-rarefaction, the traversal choose sequencing according to structure neural network model according to
The secondary parameter to be compressed for obtaining each layer.
For step S2, sub-step can be included：
S21：Build autocoding neural network based on multilayer perceptron, the input layer of autocoding neural network and
Output layer number of nodes is identical, and the number of hidden nodes is less than aobvious node layer number；
S22：Parameter to be compressed is inputted, forward conduction calculating is carried out to every layer of neuron of autocoding neural network, is obtained
To the activation value of each layer；
S23：Output is enabled to be equal to input, the residual error of output layer and each layer neuron is obtained using backward conduction algorithm；
S24：Using gradient descent method update weights W and biasing B, output is made to become closer to input；
S25：After weights and biasing convergence, the neural network parameter of the value, as low-dimensional of hidden layer is exported.
For step S3, can include：The neural network parameter of low-dimensional is solved using autocoding neural network
Pressure, compression neural network includes compression network, hidden layer reconciliation compression network, restores the quantity of neural network parameter, and will restore
Neural network parameter corresponding be placed into network.Preferably, the autocoding nerve net that step S21 is built may be used
Network unzips it, and is restored in output layer.
For the details that step S1-S3 is not specifically described, it is referred to the instruction performed by corresponding module in above device
It carries out, it will not be described here.
It is according to embodiments of the present invention in another aspect, providing a kind of compression/de-compression nerve net based on same inventive concept
The equipment of network model.
Fig. 7 is the block diagram according to the equipment of the compression/de-compression neural network model of one embodiment of the invention.The equipment
700 include：
Memory 702, for storing executable instruction；And
Processor 701, for performing the executable instruction stored in memory, to perform following operation：
Obtain the parameter to be compressed of neural network model；
The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network ginseng of low-dimensional
Number；
The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
Above-mentioned executable instruction corresponds to the corresponding steps in the above method, is to perform above method step by processor
Corresponding executable instruction.
Above-mentioned processor 701 can be single cpu (central processing unit), but can also include two or more processing
Unit.For example, processor can include general purpose microprocessor, instruction set processor and/or related chip group and/or special micro- place
Manage device (for example, application-specific integrated circuit (ASIC)).Processor can also include the onboard storage device for caching purposes.It is preferred that
, using dedicated neural network processor, and it can be multiplexed the neural network processor when instructing and performing and had
Neural network, save memory space.
Above-mentioned memory 702 can be flash memory, random access memory (RAM), read-only memory (ROM), EEPROM.It is excellent
Choosing, the on-chip storage device being mounted on chip may be used.Memory 702 can also store in addition to above-metioned instruction is stored
The neural network parameter of parameter to be compressed, low-dimensional in execution process instruction and the neural network parameter restored.
By above-described embodiment, autocoding neural network algorithm overcomes this by introducing the non-linear property of neural network
A little limitations, and export makes its result more reliable with the enforcement mechanisms of the input phase etc..Auto-encoder is a kind of unsupervised
Learning method, it represents identical meaning, the multilayer neural network with identical number of nodes using an input layer and output layer,
Learn an input and output identical " identity function ".The meaning of autocoding neural network is the most intermediate hidden layer of study,
The usual number of nodes of this layer is less compared with input layer and output layer, is the good expression of input vector.This process plays " drop
The effect of dimension " realizes that the low-dimensional of higher-dimension input represents.
In aforementioned specification, various embodiments of the present invention are described with reference to its certain exemplary embodiments.Obviously, may be used
Various modifications are made to each embodiment, and do not depart from the wider spirit and scope of the present invention described in appended claims.
Correspondingly, the description and the appended drawings should be considered illustrative and not restrictive.
Claims (11)
1. a kind of method of compression/de-compression neural network model, including step：
S1：Obtain the parameter to be compressed of neural network model；
S2：The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network parameter of low-dimensional；
S3：The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
2. according to the method described in claim 1, it is characterized in that, step S1 includes：
Traversal selection is carried out to the parameter to be compressed of neural network model, until the quantity for the parameter to be compressed chosen is equal to setting
Dimension.
3. according to the method described in claim 1, it is characterized in that, step S1 includes：
Traversal selection is carried out to the parameter to be compressed of neural network model, rarefaction is carried out to the parameter to be compressed, to choosing
Parameter to be compressed judged that the parameter to be compressed less than given threshold is arranged to 0, choose the non-zero entry after rarefaction simultaneously
The position coordinates of non-zero entry are marked, until the quantity for the parameter to be compressed chosen is equal to the dimension of setting.
4. according to the method in claim 2 or 3, which is characterized in that the traversal is chosen according to structure neural network model
Sequencing obtain the parameter to be compressed of each layer successively.
5. according to the method described in claim 1, it is characterized in that, step S2 includes sub-step：
S21：Autocoding neural network, the input layer of autocoding neural network and output are built based on multilayer perceptron
Node layer number is identical, and the number of hidden nodes is less than input layer number；
S22：Parameter to be compressed is inputted, forward conduction calculating is carried out to every layer of neuron of autocoding neural network, is obtained each
The activation value of layer；
S23：Output is enabled to be equal to input, the residual error of output layer and each layer neuron is obtained using backward conduction algorithm；
S24：Using gradient descent method update weights W and biasing B, output is made to become closer to input；
S25：After weights and biasing convergence, the neural network parameter of the value, as low-dimensional of hidden layer is exported.
6. according to the method described in claim 5, it is characterized in that, part using autocoding neural network in step S21
Network unzips it, and is restored in output layer.
7. a kind of device of compression/de-compression neural network model, is deposited including parameter acquisition module, model compression module, model
Module and model decompression module are stored up, wherein,
Parameter acquisition module, for obtaining the parameter to be compressed of neural network model；
Model compression module for compressing the parameter to be compressed using neural network algorithm, and is trained, obtains low-dimensional
Neural network parameter；
Model decompression module for decompressing the neural network parameter of low-dimensional, forms the neural network parameter of recovery；And
Memory module, for storing the parameter to be compressed of neural network model, the neural network parameter of low-dimensional and the nerve of recovery
Network parameter.
8. device according to claim 7, which is characterized in that in the model compression module, compress the ginseng to be compressed
Number is compressed by autocoding neural network algorithm, and autocoding neural network is divided into compression network, intermediate hidden layer reconciliation
Compression network, the compression network input parameter to be compressed, export to intermediate hidden layer, and the number of nodes inputted is more than the section of output
Points.
9. device according to claim 8, which is characterized in that the autocoding neural network is using multilayer perceptron as base
Plinth is built.
10. device according to claim 8, which is characterized in that in the model decompression module, decompress the nerve of low-dimensional
Network parameter is decompressed by the decompression network, and the neural network parameter of the decompression network inputs low-dimensional restores
The quantity of neural network parameter.
11. a kind of equipment of compression/de-compression neural network model, including：
Memory, for storing executable instruction；And
Processor, for performing the executable instruction stored in memory, to perform following operation：
Obtain the parameter to be compressed of neural network model；
The parameter to be compressed is compressed and trained using neural network algorithm, obtains the neural network parameter of low-dimensional；
The neural network parameter of the low-dimensional is decompressed, restores the parameter of neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611159629.6A CN108229644A (en) | 2016-12-15 | 2016-12-15 | The device of compression/de-compression neural network model, device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611159629.6A CN108229644A (en) | 2016-12-15 | 2016-12-15 | The device of compression/de-compression neural network model, device and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108229644A true CN108229644A (en) | 2018-06-29 |
Family
ID=62650420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611159629.6A Pending CN108229644A (en) | 2016-12-15 | 2016-12-15 | The device of compression/de-compression neural network model, device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108229644A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109245773A (en) * | 2018-10-30 | 2019-01-18 | 南京大学 | A kind of decoding method based on block circulation sparse matrix neural network |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
| CN110516806A (en) * | 2019-08-30 | 2019-11-29 | 苏州思必驰信息科技有限公司 | The rarefaction method and apparatus of neural network parameter matrix |
| CN110770756A (en) * | 2018-09-28 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Data processing method and device and unmanned aerial vehicle |
| CN111047020A (en) * | 2018-10-12 | 2020-04-21 | 上海寒武纪信息科技有限公司 | Neural network operation device and method supporting compression and decompression |
| CN111045726A (en) * | 2018-10-12 | 2020-04-21 | 上海寒武纪信息科技有限公司 | Deep learning processing device and method supporting coding and decoding |
| CN112261023A (en) * | 2020-10-15 | 2021-01-22 | 苏州浪潮智能科技有限公司 | Data transmission method and device of convolutional neural network |
| CN112307230A (en) * | 2019-07-29 | 2021-02-02 | 杭州海康威视数字技术股份有限公司 | Data storage method, and data acquisition method and device thereof |
| CN113168565A (en) * | 2018-12-29 | 2021-07-23 | 华为技术有限公司 | Neural network compression method and device |
| CN114731406A (en) * | 2020-12-04 | 2022-07-08 | 深圳市大疆创新科技有限公司 | Encoding method, decoding method, encoding device, and decoding device |
| WO2022147745A1 (en) * | 2021-01-08 | 2022-07-14 | 深圳市大疆创新科技有限公司 | Encoding method, decoding method, encoding apparatus, decoding apparatus |
| WO2023125934A1 (en) * | 2021-12-31 | 2023-07-06 | 维沃移动通信有限公司 | Ai network information transmission method and apparatus, and communication device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0519052A (en) * | 1991-05-08 | 1993-01-26 | Nippon Telegr & Teleph Corp <Ntt> | Recognition of three-dimensional object by neural network |
| CN104792530A (en) * | 2015-04-15 | 2015-07-22 | 北京航空航天大学 | Deep-learning rolling bearing fault diagnosis method based on SDA (stacked denoising autoencoder) and Softmax regression |
| CN105163121A (en) * | 2015-08-24 | 2015-12-16 | 西安电子科技大学 | Large-compression-ratio satellite remote sensing image compression method based on deep self-encoding network |
| CN105512175A (en) * | 2015-11-23 | 2016-04-20 | 东莞市凡豆信息科技有限公司 | Quick image retrieval method based on color features and texture characteristics |
| CN105791189A (en) * | 2016-02-23 | 2016-07-20 | 重庆大学 | Sparse coefficient decomposition method for improving reconstruction accuracy |
| CN105976408A (en) * | 2016-04-28 | 2016-09-28 | 北京大学 | Digital holographic compression transmission method of quantum backward propagation nerve network |
-
2016
- 2016-12-15 CN CN201611159629.6A patent/CN108229644A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0519052A (en) * | 1991-05-08 | 1993-01-26 | Nippon Telegr & Teleph Corp <Ntt> | Recognition of three-dimensional object by neural network |
| CN104792530A (en) * | 2015-04-15 | 2015-07-22 | 北京航空航天大学 | Deep-learning rolling bearing fault diagnosis method based on SDA (stacked denoising autoencoder) and Softmax regression |
| CN105163121A (en) * | 2015-08-24 | 2015-12-16 | 西安电子科技大学 | Large-compression-ratio satellite remote sensing image compression method based on deep self-encoding network |
| CN105512175A (en) * | 2015-11-23 | 2016-04-20 | 东莞市凡豆信息科技有限公司 | Quick image retrieval method based on color features and texture characteristics |
| CN105791189A (en) * | 2016-02-23 | 2016-07-20 | 重庆大学 | Sparse coefficient decomposition method for improving reconstruction accuracy |
| CN105976408A (en) * | 2016-04-28 | 2016-09-28 | 北京大学 | Digital holographic compression transmission method of quantum backward propagation nerve network |
Non-Patent Citations (2)
| Title |
|---|
| G. E. HINTON ETC.: ""Reducing the Dimensionality of Data with Neural Networks"", 《SCIENCE》 * |
| 杜卓明: ""三维模型检索与压缩关键技术研究"", 《中国博士学位论文全文数据库（信息科技缉）》 * |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110770756A (en) * | 2018-09-28 | 2020-02-07 | 深圳市大疆创新科技有限公司 | Data processing method and device and unmanned aerial vehicle |
| WO2020062054A1 (en) * | 2018-09-28 | 2020-04-02 | 深圳市大疆创新科技有限公司 | Data processing method and device, and unmanned aerial vehicle |
| CN111045726B (en) * | 2018-10-12 | 2022-04-15 | 上海寒武纪信息科技有限公司 | Deep learning processing device and method supporting coding and decoding |
| CN111047020B (en) * | 2018-10-12 | 2022-11-29 | 上海寒武纪信息科技有限公司 | Neural network operation device and method supporting compression and decompression |
| CN111047020A (en) * | 2018-10-12 | 2020-04-21 | 上海寒武纪信息科技有限公司 | Neural network operation device and method supporting compression and decompression |
| CN111045726A (en) * | 2018-10-12 | 2020-04-21 | 上海寒武纪信息科技有限公司 | Deep learning processing device and method supporting coding and decoding |
| CN109245773A (en) * | 2018-10-30 | 2019-01-18 | 南京大学 | A kind of decoding method based on block circulation sparse matrix neural network |
| CN109245773B (en) * | 2018-10-30 | 2021-09-28 | 南京大学 | Encoding and decoding method based on block-circulant sparse matrix neural network |
| CN113168565A (en) * | 2018-12-29 | 2021-07-23 | 华为技术有限公司 | Neural network compression method and device |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
| CN109978142B (en) * | 2019-03-29 | 2022-11-29 | 腾讯科技（深圳）有限公司 | Neural network model compression method and device |
| CN112307230A (en) * | 2019-07-29 | 2021-02-02 | 杭州海康威视数字技术股份有限公司 | Data storage method, and data acquisition method and device thereof |
| CN112307230B (en) * | 2019-07-29 | 2024-01-26 | 杭州海康威视数字技术股份有限公司 | Data storage method, data acquisition method and device |
| CN110516806A (en) * | 2019-08-30 | 2019-11-29 | 苏州思必驰信息科技有限公司 | The rarefaction method and apparatus of neural network parameter matrix |
| CN112261023A (en) * | 2020-10-15 | 2021-01-22 | 苏州浪潮智能科技有限公司 | Data transmission method and device of convolutional neural network |
| CN114731406A (en) * | 2020-12-04 | 2022-07-08 | 深圳市大疆创新科技有限公司 | Encoding method, decoding method, encoding device, and decoding device |
| WO2022147745A1 (en) * | 2021-01-08 | 2022-07-14 | 深圳市大疆创新科技有限公司 | Encoding method, decoding method, encoding apparatus, decoding apparatus |
| WO2023125934A1 (en) * | 2021-12-31 | 2023-07-06 | 维沃移动通信有限公司 | Ai network information transmission method and apparatus, and communication device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229644A (en) | The device of compression/de-compression neural network model, device and method | |
| CN107767408B (en) | Image processing method, processing device and processing equipment | |
| EP3522079B1 (en) | Data encoding and classification | |
| Gregor et al. | Towards conceptual compression | |
| CN113111760B (en) | Light-weight graph convolution human skeleton action recognition method based on channel attention | |
| JP6127219B2 (en) | Method and system for extracting facial features from facial image data | |
| CN107688855A (en) | It is directed to the layered quantization method and apparatus of Complex Neural Network | |
| CN104050507B (en) | Hyperspectral image classification method based on multilayer neural network | |
| CN104918046B (en) | A kind of local description compression method and device | |
| CN108108807A (en) | Learning-oriented image processing method, system and server | |
| CN108171663A (en) | The image completion system for the convolutional neural networks that feature based figure arest neighbors is replaced | |
| CN107679572A (en) | A kind of image discriminating method, storage device and mobile terminal | |
| CN106686385A (en) | Video compression sensing reconstruction method and device | |
| Araújo et al. | Local adaptive receptive field self-organizing map for image color segmentation | |
| CN110070583A (en) | Signal compression and restoration methods and system based on tensor resolution and deep learning | |
| CN107944488B (en) | Long time series data processing method based on stratification depth network | |
| Gupta et al. | Hybrid image compression-encryption scheme based on multilayer stacked autoencoder and logistic map | |
| CN106169961A (en) | The network parameter processing method and processing device of neutral net based on artificial intelligence | |
| CN115600686A (en) | Personalized Transformer-based federal learning model training method and federal learning system | |
| CN110598585A (en) | Sit-up action recognition method based on convolutional neural network | |
| CN113947119A (en) | Method for detecting human gait by using plantar pressure signals | |
| Zeman et al. | Compressed superposition of neural networks for deep learning in edge computing | |
| CN110782503A (en) | Face image synthesis method and device based on two-branch depth correlation network | |
| CN115984949A (en) | Low-quality face image recognition method and device with attention mechanism | |
| CN115546325A (en) | Image compressed sensing self-adaptive reconstruction method and device based on textural features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180629 |
|
| RJ01 | Rejection of invention patent application after publication |