专栏首页机器学习、深度学习网络模型--Densely Connected Convolutional Networks

网络模型--Densely Connected Convolutional Networks

Densely Connected Convolutional Networks CVPR2017 best paper Code: https://github.com/liuzhuang13/DenseNet

本文受到 ResNet and Highway Networks 的启发: bypass signal from one layer to the next via identity connections,这里主要是多加了几个 identity connections,发现这么干效果很好。

首先看看 一个 5层 的Dense Block 是怎么Densely Connected

上面5层的模块有多少连接了? 5*(5+1)/2=15

整个网络结构如下图所示:

  1. DenseNets ResNets [11] add a skip-connection that bypasses the non-linear transformations with an identity function

Dense connectivity

where [x_0 ,x _1 ,…,x_l−1 ] refers to the concatenation of the feature-maps produced in layers 0,…,l−1

Composite function H定义为 a composite function of three consecutive operations: batch normalization (BN) , followed by a rectified linear unit (ReLU) and a 3 × 3 convolution (Conv)

Pooling layers 可以改变特征图的尺寸,便于 concatenation

Growth rate:If each function H produces k feature-maps as output,We refer to the hyper-parameter k as the growth rate of the network

Bottleneck layers:尽管每个网络层只输出 k 个特征图,但是同时仍然有太多的输入个数,通常的做法是降维,在进行3×3卷积之前首先用一个 1×1卷积将输入个数降低到 4*k, 也就是在 H的定义中再加入一个 1×1卷积 Although each layer only produces k output feature maps, it typically has many more inputs. It has been noted in [36, 11] that a 1×1 convolution can be introduced as bottleneck layer before each 3×3 convolution to reduce the number of input feature-maps

为什么有太多的输入个数了? If each function H_l produces k feature-maps as output, it follows that the l th layer has k×(l−1)+k0 input feature-maps, where k 0 is the number of channels in the input image.

Compression :为了进一步提升模型的简洁性,我们在 transition layers里 降低特征图数量 To further improve model compactness, we can reduce the number of feature-maps at transition layers. If a dense block contains m feature-maps, we let the following transition layer generate bθmc output feature-maps, where 0 <θ ≤1 is referred to as the compression factor.

  1. Experiments Error rates (%) on CIFAR and SVHN datasets

DenseNet and ResNet Top-1 (single model and single-crop) 对比:

参数规模还是比较小的

1) Middle: DenseNet-BC requires about 1/3 of the parameters as ResNet to achieve comparable accuracy 2)Right: Training and testing curves of the 1001-layer pre-activation ResNet [12] with more than 10M parameters and a 100-layer DenseNet with only 0.8M parameters

总的来说就是简单的多加几个 shortcut ,效果就好了,计算量少了!

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • 相机模型--A Theory of Catadioptric Image Formation

    版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.n...

    用户1148525
  • 人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding

    Slicing Convolutional Neural Network for Crowd Video Understanding CVPR2016 h...

    用户1148525
  • 视频物体分割--One-Shot Video Object Segmentation

    One-Shot Video Object Segmentation CVPR2017 http://www.vision.ee.ethz.ch/~cvl...

    用户1148525
  • 10分钟上手,OpenCV自然场景文本检测(Python代码+实现)

    EAST文本检测器需要OpenCV3.4.2或更高版本,有需要的读者可以先安装OpenCV。

    磐创AI
  • UVa Automatic Editing

    uva的题真的很好,每个题都能长许多知识,A了后很开心,这道题我用了两天写,只一道题就学了四个函数,成长不少 Problem E: Automatic Edit...

    用户1624346
  • TensorFlow GPU 版安装

    木东居士
  • PAT 1007

    Given a sequence of K integers { N1, N2, ..., NK }. A continuous subsequence is...

    week
  • 呕心沥血倾力巨制T-lnP图攻略——奥斯陆的气象生活

    搞这个东西的初衷是因为我自己学的时候也被搞的很烦,而且概念乱七八糟,脑子里一团浆糊,一方面方便自己一方面也方便各位大气学子。本攻略以本题为例:

    气象学家
  • 10分钟上手,OpenCV自然场景文本检测(Python代码+实现)

    EAST文本检测器需要OpenCV3.4.2或更高版本,有需要的读者可以先安装OpenCV。

    新智元
  • 2019-2020 ICPC Southeastern European Regional Programming Contest (SEERC 2019)-G.Projection

    Everybody knows that you are a TensorFlow fan. Therefore, you’ve been challenged...

    某些人

扫码关注云+社区

领取腾讯云代金券