CN110097177B - 一种基于伪孪生网络的网络剪枝方法 - Google Patents
一种基于伪孪生网络的网络剪枝方法 Download PDFInfo
- Publication number
- CN110097177B CN110097177B CN201910400920.5A CN201910400920A CN110097177B CN 110097177 B CN110097177 B CN 110097177B CN 201910400920 A CN201910400920 A CN 201910400920A CN 110097177 B CN110097177 B CN 110097177B
- Authority
- CN
- China
- Prior art keywords
- network
- pseudo
- twin
- target
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000013138 pruning Methods 0.000 title claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 12
- 238000007906 compression Methods 0.000 abstract description 7
- 230000006835 compression Effects 0.000 abstract description 7
- 238000013528 artificial neural network Methods 0.000 abstract description 4
- 238000013140 knowledge distillation Methods 0.000 abstract description 4
- 238000003062 neural network model Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 6
- 238000009966 trimming Methods 0.000 description 5
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000013139 quantization Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 208000029028 brain injury Diseases 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
Description
网络层 | 参数量 | 剪枝率 |
Conv1 | 150 | 0.413 |
Conv2 | 2400 | 0.467 |
Conv3 | 48000 | 0.528 |
FC1 | 10080 | 0.401 |
FC2 | 840 | 0.407 |
Total | 61470 | 0.503 |
模型名 | 错误率 |
LeNet | 1.39% |
三值值的LeNet | 1.97% |
裁剪后的LeNet | 1.49% |
网络层 | 参数量 | 剪枝率 |
Conv1 | 150 | 0.405 |
Conv2 | 2400 | 0.201 |
Conv3 | 48000 | 0.413 |
FC1 | 10080 | 0.436 |
FC2 | 840 | 0.423 |
Total | 61470 | 0.408 |
模型名 | 错误率 |
LeNet | 1.39% |
三值的LeNet | 1.97% |
裁剪后的LeNet | 1.37% |
Model | Error Rate |
AlexNet | 0.232 |
Ternary AlexNet | 0.253 |
Pruned AlexNet | 0.181 |
Layer | Params | Compression Rate |
Conv1 | 4800 | 40.4% |
Conv2 | 153600 | 39.4% |
Conv3 | 110592 | 40.5% |
Conv4 | 147456 | 41.2% |
Conv5 | 147456 | 40.0% |
Fc1 | 4718592 | 58.0% |
Fc2 | 16777216 | 62.6% |
Fc3 | 40960 | 51.2% |
Total | 22100672 | 62.0% |
Claims (3)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910400920.5A CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910400920.5A CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110097177A CN110097177A (zh) | 2019-08-06 |
CN110097177B true CN110097177B (zh) | 2022-11-29 |
Family
ID=67448052
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910400920.5A Active CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110097177B (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210150313A1 (en) * | 2019-11-15 | 2021-05-20 | Samsung Electronics Co., Ltd. | Electronic device and method for inference binary and ternary neural networks |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110909667B (zh) * | 2019-11-20 | 2022-05-10 | 北京化工大学 | 面向多角度sar目标识别网络的轻量化设计方法 |
CN111091144B (zh) * | 2019-11-27 | 2023-06-27 | 云南电网有限责任公司电力科学研究院 | 基于深度伪孪生网络的图像特征点匹配方法及装置 |
CN111008693B (zh) * | 2019-11-29 | 2024-01-26 | 小米汽车科技有限公司 | 一种基于数据压缩的网络模型构建方法、***和介质 |
CN111695699B (zh) * | 2020-06-12 | 2023-09-08 | 北京百度网讯科技有限公司 | 用于模型蒸馏的方法、装置、电子设备及可读存储介质 |
CN112348167B (zh) * | 2020-10-20 | 2022-10-11 | 华东交通大学 | 一种基于知识蒸馏的矿石分选方法和计算机可读存储介质 |
CN113724261A (zh) * | 2021-08-11 | 2021-11-30 | 电子科技大学 | 一种基于卷积神经网络的快速图像构图方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108334934A (zh) * | 2017-06-07 | 2018-07-27 | 北京深鉴智能科技有限公司 | 基于剪枝和蒸馏的卷积神经网络压缩方法 |
CN109543559A (zh) * | 2018-10-31 | 2019-03-29 | 东南大学 | 基于孪生网络和动作选择机制的目标跟踪方法及*** |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10740676B2 (en) * | 2016-05-19 | 2020-08-11 | Nec Corporation | Passive pruning of filters in a convolutional neural network |
-
2019
- 2019-05-15 CN CN201910400920.5A patent/CN110097177B/zh active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108334934A (zh) * | 2017-06-07 | 2018-07-27 | 北京深鉴智能科技有限公司 | 基于剪枝和蒸馏的卷积神经网络压缩方法 |
CN109543559A (zh) * | 2018-10-31 | 2019-03-29 | 东南大学 | 基于孪生网络和动作选择机制的目标跟踪方法及*** |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20210150313A1 (en) * | 2019-11-15 | 2021-05-20 | Samsung Electronics Co., Ltd. | Electronic device and method for inference binary and ternary neural networks |
Also Published As
Publication number | Publication date |
---|---|
CN110097177A (zh) | 2019-08-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110097177B (zh) | 一种基于伪孪生网络的网络剪枝方法 | |
WO2021042828A1 (zh) | 神经网络模型压缩的方法、装置、存储介质和芯片 | |
CN111079781B (zh) | 基于低秩与稀疏分解的轻量化卷积神经网络图像识别方法 | |
CN113159173A (zh) | 一种结合剪枝与知识蒸馏的卷积神经网络模型压缩方法 | |
CN111695513B (zh) | 一种基于深度残差网络的人脸表情识别方法 | |
Yue et al. | Face recognition based on histogram equalization and convolution neural network | |
CN112418397B (zh) | 一种基于轻量级卷积神经网络的图像分类方法 | |
CN111784699B (zh) | 一种对三维点云数据进行目标分割方法、装置及终端设备 | |
Singh et al. | Acceleration of deep convolutional neural networks using adaptive filter pruning | |
CN115829027A (zh) | 一种基于对比学习的联邦学习稀疏训练方法及*** | |
CN112597919A (zh) | 基于YOLOv3剪枝网络和嵌入式开发板的实时药盒检测方法 | |
Fan et al. | HFPQ: deep neural network compression by hardware-friendly pruning-quantization | |
Qi et al. | Learning low resource consumption cnn through pruning and quantization | |
Doan | Large-scale insect pest image classification | |
CN114742997A (zh) | 一种面向图像分割的全卷积神经网络密度峰剪枝方法 | |
CN114882278A (zh) | 一种基于注意力机制和迁移学习的轮胎花纹分类方法和装置 | |
CN110807497A (zh) | 基于深度动态网络的手写体数据分类方法及*** | |
CN117671271A (zh) | 模型训练方法、图像分割方法、装置、设备和介质 | |
CN112561054A (zh) | 一种基于批量特征热图的神经网络滤波器剪枝方法 | |
CN116957010A (zh) | 一种用于卷积神经网络的模型推理方法和装置 | |
CN116310335A (zh) | 一种基于Vision Transformer的翼状胬肉病灶区域的分割方法 | |
CN115620064A (zh) | 一种基于卷积神经网络的点云下采样分类方法及*** | |
CN115063374A (zh) | 模型训练、人脸图像质量评分方法、电子设备及存储介质 | |
CN114494284B (zh) | 一种基于显式监督区域关系的场景解析模型及方法 | |
CN115100694A (zh) | 一种基于自监督神经网络的指纹快速检索方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20240607 Address after: Room 24, Floor 2, Unit 1, Building 1, No. 73, Section 2, Second Ring Road West, Qingyang District, Chengdu, 610000, Sichuan Patentee after: Aegis Defense Technology (Chengdu) Co.,Ltd. Country or region after: China Address before: 610041 floor 5, building 1, No. 21, Gaopeng Avenue, high tech Zone, Chengdu, Sichuan Patentee before: Electric Coreda (Chengdu) Technology Co.,Ltd. Country or region before: China |