Trained Ternary Quantization ICLR 2017 https://github.com/TropComplique/trained-ternary-quantization pytorch https://github.com/buaabai/Ternary-Weights-Network pytorch
传统的二值网络将权重 W 量化为 +1、-1; 三值网络 TWN (Ternary weight networks) 将权重W 量化为 {−W_l ,0,+W_l }
阈值的计算公式如下所示
本文提出了新的三值网络
positive and negative weights,三个不同的值用于表示三值网络,这个正负权值是通过网络学习得到的 对应的梯度计算如下
本文的阈值选择采用:
set t to 0.05 in experiments on CIFAR-10 and ImageNet dataset
The quantization roughly proceeds as follows.