不可不知的卷积神经网络调参技巧-李沐大神的私房调参笔记

摘要：最近在图像分类研究中取得的许多进展可以归功于训练过程的改进，例如数据增强和优化方法的改变。然而，大多数改进要么只是作为实现细节在文献中被简要地提到，要么只在源代码中才能见到。在本文中，我们将对这一系列改进进行测试，并通过对比试验评估它们对最终模型精度的影响。结果表明，通过合理的将这些改进组合在一起，能够显著改进各种CNN模型性能。例如，将ResNet-50在ImageNet上的top-1验证精度从75.3%提高到79.29%。同时，图像分类准确性的提高将有助于提升在其他应用领域如目标检测和语义分割的迁移学习性能。

Bag of Tricks for Image Classification with Convolutional Neural Networks

Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li

AbstractMuch of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods. In the literature, however, most refinements are either briefly mentioned as implementation details or only visible in source code. In this paper, we will examine a collection of such refinements and empirically evaluate their impact on the final model accuracy through ablation study. We will show that, by combining these refinements together, we are able to improve various CNN models significantly. For example, we raise ResNet-50's top-1 validation accuracy from 75.3% to 79.29% on ImageNet. We will also demonstrate that improvement on image classification accuracy leads to better transfer learning performance in other application domains such as object detection and semantic segmentation.