基于Pytorch构建GoogLeNet网络对cifar-10进行分类

GoogLeNet是2014年Christian Szegedy提出的一种全新的深度学习结构，在这之前的AlexNet、VGG等结构都是通过增大网络的深度（层数）来获得更好的训练效果，但层数的增加会带来很多负作用，比如overfit、梯度消失、梯度爆炸等。inception的提出则从另一种角度来提升训练结果：能更高效的利用计算资源，在相同的计算量下能提取到更多的特征，从而提升训练结果。

inception模块的基本机构如图所示，整个inception结构就是由多个这样的inception模块串联起来的。inception结构的主要贡献有两个：一是使用1x1的卷积来进行升降维；二是在多个尺寸上同时进行卷积再聚合。

1、引入Inception结构

引入的Inception融合了不同尺度的特征信息，能得到更好的特征表征。

更意味着提高准确率，不一定需要堆叠更深的层或者增加神经元个数等，可以转向研究更稀疏但是更精密的结构同样可以达到很好的效果。

2、使用1x1的卷积核进行降维映射处理

降低了维度也减少了参数量（NiN是用于代替全连接层）。

3、添加两个辅助分类器帮助训练

避免梯度消失，用于向前传导梯度，也有一定的正则化效果，防止过拟合。

4、使用全局平均池化

用全局平均池化代替全连接层大大减少了参数量（与NiN一致）

5、1*n和n*1卷积核并联代替n*n卷积核

在InceptionV3中，在不改变感受野同时减少参数的情况下，采用1*n和n*1的卷积核并联来代替InceptionV1-V2中n*n的卷积核（发掘特征图的高的特征，以及特征图的宽的特征）

具体代码如下，因为里面的层数太多，所以在此就不做推算了：

1. classGoogLeNet(nn.Module):
2. def __init__(self, num_classes=10, aux_logits=False):
3. super().__init__()
4. self.aux_logits = aux_logits
6. self.conv1 = BasicConv2d(3, 64, kernel_size=3, padding=1)
7. self.maxpool1 = nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True)
9. self.conv2 = BasicConv2d(64, 192, kernel_size=3, padding=1)
10. self.maxpool2 = nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True)
12. self.inception3a = Inception(192, 64, 96, 128, 16, 32, 32)
14. self.inception3b = Inception(256, 128, 128, 192, 32, 96, 64)
15. self.maxpool3 = nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True)
17. self.inception4a = Inception(480, 192, 96, 208, 16, 48, 64)
18. self.inception4b = Inception(512, 160, 112, 224, 24, 64, 64)
19. self.inception4c = Inception(512, 128, 128, 256, 24, 64, 64)
20. self.inception4d = Inception(512, 112, 144, 288, 32, 64, 64)
21. self.inception4e = Inception(528, 256, 160, 320, 32, 128, 128)
22. self.maxpool4 = nn.MaxPool2d(kernel_size=3, stride=2, ceil_mode=True)
24. self.inception5a = Inception(832, 256, 160, 320, 32, 128, 128)
25. self.inception5b = Inception(832, 384, 192, 384, 48, 128, 128)
27. if self.aux_logits:
28. self.aux1 = AuxClassifier(512, num_classes)
29. self.aux2 = AuxClassifier(528, num_classes)
31. self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
32. self.dropout = nn.Dropout(0.4)
33. self.fc = nn.Linear(1024, num_classes)
35. for m in self.modules():
36. if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):
37. nn.init.kaiming_uniform_(m.weight, mode='fan_out', nonlinearity='relu')
39. def forward(self, X):
40. X = self.conv1(X)
41. X = self.conv2(X)
43. X = self.inception3a(X)
44. X = self.inception3b(X)
45. X = self.maxpool3(X)
47. X = self.inception4a(X)
48. if self.training and self.aux_logits:
49. aux1 = self.aux1(X)
50. X = self.inception4b(X)
51. X = self.inception4c(X)
52. X = self.inception4d(X)
53. if self.training and self.aux_logits:
54. aux2 = self.aux2(X)
55. X = self.inception4e(X)
56. X = self.maxpool4(X)
58. X = self.inception5a(X)
59. X = self.inception5b(X)
61. X = self.avgpool(X)
62. X = torch.flatten(X, start_dim=1)
63. X = self.dropout(X)
64. X = self.fc(X)
66. if self.training and self.aux_logits:
67. return X, aux2, aux1
68. return X
71. classInception(nn.Module):
72. def __init__(self, in_channels, ch1x1, ch3x3red, ch3x3, ch5x5red, ch5x5, pool_proj):
73. super().__init__()
74. print('Inception params:',in_channels, ch1x1, ch3x3red, ch3x3, ch5x5red, ch5x5, pool_proj)
76. self.branch1 = BasicConv2d(in_channels, ch1x1, kernel_size=1)
77. print('branch1 params:', in_channels, ch1x1)
79. self.branch2 = nn.Sequential(
80. BasicConv2d(in_channels, ch3x3red, kernel_size=1),
81. BasicConv2d(ch3x3red, ch3x3, kernel_size=3, padding=1)
82. )
83. print('branch2 params:', in_channels, ch3x3red,ch3x3)
85. self.branch3 = nn.Sequential(
86. BasicConv2d(in_channels, ch5x5red, kernel_size=1),
87. BasicConv2d(ch5x5red, ch5x5, kernel_size=5, padding=2)
88. )
89. print('branch3 params:', in_channels, ch5x5red,ch5x5)
91. self.branch4 = nn.Sequential(
92. nn.MaxPool2d(kernel_size=3, stride=1, padding=1),
93. BasicConv2d(in_channels, pool_proj, kernel_size=1)
94. )
95. print('branch4 params:', in_channels, pool_proj)
97. def forward(self, X):
98. branch1 = self.branch1(X)
99. branch2 = self.branch2(X)
100. branch3 = self.branch3(X)
101. branch4 = self.branch4(X)
103. outputs = [branch1, branch2, branch3, branch4]
104. return torch.cat(outputs, dim=1)
107. classAuxClassifier(nn.Module):
108. def __init__(self, in_channels, num_classes):
109. super().__init__()
110. self.averagePool = nn.AvgPool2d(kernel_size=5, stride=3)
111. self.conv = BasicConv2d(in_channels, 128, kernel_size=1)
113. self.fc1 = nn.Linear(2048, 1024)
114. self.fc2 = nn.Linear(1024, num_classes)
116. def forward(self, X):
117. X = self.averagePool(X)
118. X = self.conv(X)
120. X = torch.flatten(X, start_dim=1)
122. X = F.relu(self.fc1(X), inplace=True)
123. X = F.dropout(X, 0.7, training=self.training)
124. X = self.fc2(X)
126. return X
129. classBasicConv2d(nn.Module):
130. def __init__(self, in_channels, out_channels, **kwargs):
131. super().__init__()
132. self.conv = nn.Conv2d(in_channels, out_channels, **kwargs)
133. print('BasicConv2d params:',in_channels, out_channels)
134. self.bn = nn.BatchNorm2d(out_channels)
135. self.relu = nn.ReLU(inplace=True)
137. def forward(self, X):
138. return self.relu(self.bn(self.conv(X)))

下图是打印出来的执行顺序，便于消化理解GoogLeNet网络。

下面是运行的情况以及训练集和验证集的准确率，GoogLeNet较之前VGG-16网络运行时间大大增加，而且在CPU笔记本上基本无法运行了，此外准确率较VGG-16并没有看出提升，训练了100轮准确率也只有79%，还比不上之前VGG-16网络。

1. start_time 2023-08-1817:12:02
2. TrainEpoch1Loss: 12.900574, accuracy: 9.375000%
3. test_avarage_loss: 0.043172, accuracy: 46.720000%
4. end_time: 2023-08-1817:15:01
5. ...
7. start_time 2023-08-1818:08:22
8. TrainEpoch20Loss: 0.188147, accuracy: 95.312500%
9. test_avarage_loss: 0.016367, accuracy: 75.200000%
10. end_time: 2023-08-1818:11:19
12. ...
14. start_time 2023-08-1822:04:30
15. TrainEpoch100Loss: 0.007709, accuracy: 100.000000%
16. test_avarage_loss: 0.023382, accuracy: 79.360000%
17. end_time: 2023-08-1822:07:27

下面是代码里输出的损失率和准确率

这是基于深度学习开展图像识别的第四个模型，估计是一些参数未做调优导致的，接下来会尝试一下ResNet网络。