量化网络训练--Towards Effective Low-bitwidth Convolutional Neural Networks

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://cloud.tencent.com/developer/article/1436549

Towards Effective Low-bitwidth Convolutional Neural Networks

CVPR2018

https://github.com/nowgood/QuantizeCNNModel

本文针对 低 bitwidth CNN网络 提出三个训练技巧 以得到较高精度。这些技巧可以独立使用也可以结合使用。

第一个技巧：首先量化 weight，得到足够好的效果后再 量化 activation

第二个技巧：逐步降低网络的位数，32-bit→8-bit→4-bit→2-bit

第三个技巧：同时训练高精度网络和低精度网络，两者相互学习 train a full-precision network alongside the target low-precision network， Guided training with a full-precision network

It is observed that by using the guidance of the teacher model, better performance can be obtained with the student model than directly training the student model on the target problem.