CN110097177A - 一种基于伪孪生网络的网络剪枝方法 - Google Patents
一种基于伪孪生网络的网络剪枝方法 Download PDFInfo
- Publication number
- CN110097177A CN110097177A CN201910400920.5A CN201910400920A CN110097177A CN 110097177 A CN110097177 A CN 110097177A CN 201910400920 A CN201910400920 A CN 201910400920A CN 110097177 A CN110097177 A CN 110097177A
- Authority
- CN
- China
- Prior art keywords
- network
- pseudo
- twin
- weight
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013138 pruning Methods 0.000 title claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 18
- 238000011478 gradient descent method Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 12
- 238000007906 compression Methods 0.000 abstract description 8
- 230000006835 compression Effects 0.000 abstract description 8
- 238000013528 artificial neural network Methods 0.000 abstract description 4
- 238000010276 construction Methods 0.000 abstract description 2
- 238000003062 neural network model Methods 0.000 abstract description 2
- 238000007429 general method Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000009966 trimming Methods 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000013139 quantization Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000013140 knowledge distillation Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 208000022306 Cerebral injury Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000004821 distillation Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
Description
网络层 | 参数量 | 剪枝率 |
Conv1 | 150 | 0.413 |
Conv2 | 2400 | 0.467 |
Conv3 | 48000 | 0.528 |
FC1 | 10080 | 0.401 |
FC2 | 840 | 0.407 |
Total | 61470 | 0.503 |
模型名 | 错误率 |
LeNet | 1.39% |
三值值的LeNet | 1.97% |
裁剪后的LeNet | 1.49% |
网络层 | 参数量 | 剪枝率 |
Conv1 | 150 | 0.405 |
Conv2 | 2400 | 0.201 |
Conv3 | 48000 | 0.413 |
FC1 | 10080 | 0.436 |
FC2 | 840 | 0.423 |
Total | 61470 | 0.408 |
模型名 | 错误率 |
LeNet | 1.39% |
三值的LeNet | 1.97% |
裁剪后的LeNet | 1.37% |
Model | Error Rate |
AlexNet | 0.232 |
Ternary AlexNet | 0.253 |
Pruned AlexNet | 0.181 |
Layer | Params | Compression Rate |
Conv1 | 4800 | 40.4% |
Conv2 | 153600 | 39.4% |
Conv3 | 110592 | 40.5% |
Conv4 | 147456 | 41.2% |
Conv5 | 147456 | 40.0% |
Fc1 | 4718592 | 58.0% |
Fc2 | 16777216 | 62.6% |
Fc3 | 40960 | 51.2% |
Total | 22100672 | 62.0% |
Claims (3)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910400920.5A CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910400920.5A CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110097177A true CN110097177A (zh) | 2019-08-06 |
CN110097177B CN110097177B (zh) | 2022-11-29 |
Family
ID=67448052
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910400920.5A Active CN110097177B (zh) | 2019-05-15 | 2019-05-15 | 一种基于伪孪生网络的网络剪枝方法 |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110097177B (zh) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110909667A (zh) * | 2019-11-20 | 2020-03-24 | 北京化工大学 | 面向多角度sar目标识别网络的轻量化设计方法 |
CN111008693A (zh) * | 2019-11-29 | 2020-04-14 | 深动科技(北京)有限公司 | 一种基于数据压缩的网络模型构建方法、***和介质 |
CN111091144A (zh) * | 2019-11-27 | 2020-05-01 | 云南电网有限责任公司电力科学研究院 | 基于深度伪孪生网络的图像特征点匹配方法及装置 |
CN111695699A (zh) * | 2020-06-12 | 2020-09-22 | 北京百度网讯科技有限公司 | 用于模型蒸馏的方法、装置、电子设备及可读存储介质 |
CN112348167A (zh) * | 2020-10-20 | 2021-02-09 | 华东交通大学 | 一种基于知识蒸馏的矿石分选方法和计算机可读存储介质 |
CN113724261A (zh) * | 2021-08-11 | 2021-11-30 | 电子科技大学 | 一种基于卷积神经网络的快速图像构图方法 |
US12039430B2 (en) * | 2019-11-15 | 2024-07-16 | Samsung Electronics Co., Ltd. | Electronic device and method for inference binary and ternary neural networks |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170337472A1 (en) * | 2016-05-19 | 2017-11-23 | Nec Laboratories America, Inc. | Passive pruning of filters in a convolutional neural network |
CN108334934A (zh) * | 2017-06-07 | 2018-07-27 | 北京深鉴智能科技有限公司 | 基于剪枝和蒸馏的卷积神经网络压缩方法 |
CN109543559A (zh) * | 2018-10-31 | 2019-03-29 | 东南大学 | 基于孪生网络和动作选择机制的目标跟踪方法及*** |
-
2019
- 2019-05-15 CN CN201910400920.5A patent/CN110097177B/zh active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170337472A1 (en) * | 2016-05-19 | 2017-11-23 | Nec Laboratories America, Inc. | Passive pruning of filters in a convolutional neural network |
CN108334934A (zh) * | 2017-06-07 | 2018-07-27 | 北京深鉴智能科技有限公司 | 基于剪枝和蒸馏的卷积神经网络压缩方法 |
CN109543559A (zh) * | 2018-10-31 | 2019-03-29 | 东南大学 | 基于孪生网络和动作选择机制的目标跟踪方法及*** |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US12039430B2 (en) * | 2019-11-15 | 2024-07-16 | Samsung Electronics Co., Ltd. | Electronic device and method for inference binary and ternary neural networks |
CN110909667A (zh) * | 2019-11-20 | 2020-03-24 | 北京化工大学 | 面向多角度sar目标识别网络的轻量化设计方法 |
CN111091144A (zh) * | 2019-11-27 | 2020-05-01 | 云南电网有限责任公司电力科学研究院 | 基于深度伪孪生网络的图像特征点匹配方法及装置 |
CN111008693A (zh) * | 2019-11-29 | 2020-04-14 | 深动科技(北京)有限公司 | 一种基于数据压缩的网络模型构建方法、***和介质 |
CN111008693B (zh) * | 2019-11-29 | 2024-01-26 | 小米汽车科技有限公司 | 一种基于数据压缩的网络模型构建方法、***和介质 |
CN111695699A (zh) * | 2020-06-12 | 2020-09-22 | 北京百度网讯科技有限公司 | 用于模型蒸馏的方法、装置、电子设备及可读存储介质 |
CN111695699B (zh) * | 2020-06-12 | 2023-09-08 | 北京百度网讯科技有限公司 | 用于模型蒸馏的方法、装置、电子设备及可读存储介质 |
CN112348167A (zh) * | 2020-10-20 | 2021-02-09 | 华东交通大学 | 一种基于知识蒸馏的矿石分选方法和计算机可读存储介质 |
CN112348167B (zh) * | 2020-10-20 | 2022-10-11 | 华东交通大学 | 一种基于知识蒸馏的矿石分选方法和计算机可读存储介质 |
CN113724261A (zh) * | 2021-08-11 | 2021-11-30 | 电子科技大学 | 一种基于卷积神经网络的快速图像构图方法 |
Also Published As
Publication number | Publication date |
---|---|
CN110097177B (zh) | 2022-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110097177A (zh) | 一种基于伪孪生网络的网络剪枝方法 | |
CN110766063B (zh) | 基于压缩激励和紧密连接卷积神经网络的图像分类方法 | |
CN111242288B (zh) | 一种用于病变图像分割的多尺度并行深度神经网络模型构建方法 | |
CN110070107A (zh) | 物体识别方法及装置 | |
WO2021051987A1 (zh) | 神经网络模型训练的方法和装置 | |
CN106383891A (zh) | 一种基于深度哈希的医学图像分布式检索方法 | |
CN110135231A (zh) | 动物面部识别方法、装置、计算机设备和存储介质 | |
JP2023523029A (ja) | 画像認識モデル生成方法、装置、コンピュータ機器及び記憶媒体 | |
CN110929610A (zh) | 基于cnn模型和迁移学习的植物病害识别方法及*** | |
CN110309853A (zh) | 基于变分自编码器的医学图像聚类方法 | |
CN109271516A (zh) | 一种知识图谱中实体类型分类方法及*** | |
Zhu et al. | Big data image classification based on distributed deep representation learning model | |
Du et al. | Efficient network construction through structural plasticity | |
CN116779091A (zh) | 一种多模态网络互联融合的胸部影像诊断报告自动生成方法 | |
CN111914166A (zh) | 应用于社区矫正人员的矫正策略个性化推荐*** | |
Qi et al. | Learning low resource consumption cnn through pruning and quantization | |
CN114580517A (zh) | 一种图像识别模型的确定方法及装置 | |
CN108875909A (zh) | 医学影像的分类方法及装置 | |
CN116797850A (zh) | 基于知识蒸馏和一致性正则化的类增量图像分类方法 | |
CN115620064A (zh) | 一种基于卷积神经网络的点云下采样分类方法及*** | |
CN115063374A (zh) | 模型训练、人脸图像质量评分方法、电子设备及存储介质 | |
CN116011550A (zh) | 一种模型剪枝方法、图像处理方法及相关装置 | |
CN105184320A (zh) | 基于结构相似度的非负稀疏编码的图像分类方法 | |
CN115455162A (zh) | 层次胶囊与多视图信息融合的答案句子选择方法与装置 | |
CN116012903A (zh) | 一种人脸表情自动标注的方法及*** |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20240607 Address after: Room 24, Floor 2, Unit 1, Building 1, No. 73, Section 2, Second Ring Road West, Qingyang District, Chengdu, 610000, Sichuan Patentee after: Aegis Defense Technology (Chengdu) Co.,Ltd. Country or region after: China Address before: 610041 floor 5, building 1, No. 21, Gaopeng Avenue, high tech Zone, Chengdu, Sichuan Patentee before: Electric Coreda (Chengdu) Technology Co.,Ltd. Country or region before: China |
|
TR01 | Transfer of patent right |