首页
学习
活动
专区
工具
TVP
发布

CNN

原创
作者头像
用户4880117
修改2019-03-19 09:28:06
1K0
修改2019-03-19 09:28:06
举报
文章被收录于专栏:DEEPLEARNDEEPLEARN

目录

1.lenet5网络结构
2.AlexNet网络结构
3.VGGNet网络结构
4.InterceptNet网络结构
5.ResNet网络结构

—————————————————————

1.lenet5网络结构
  • **(1)Lenet5**

conv1->pool1(sigmoid)->

conv2->pool2(sigmoid)->

Fc1->Fc2->Fc3

32x32x1(f=5,s=1)—>28x28x6(k=2,s=2)—>14x14x6(f=5,s=1)—>10x10x16(k=2,s=2)—>5x5x16—>Fc(120)—>Fc(84)—>10

  • **(2)Param**

Input: x = None,32,32,1

Conv1: w = 5,5,1,6 ; stride=1,1,1,1

Feature1: net = None,28,28,6 ; Param=5x5+1x6

Feature_p: None,14,14,6

Conv2: w = [5,5,6,16]  ; stride=[1,1,1,1]

Feature1: net = None,10,10,16 ; Param=6x(3x25+1)+6x(4x25+1)+3x(4x25+1)+(25x6+1)=1516

Pool2 : ksize=1,2,2,1

Feature_p: None,5,5,16

Fc_1: w = 5,5,16,120 ; stride=1,1,1,1

Feature3: net = None,1,1,120 ; Param=120x(16x5x5+1)= 48120

Fc_2: Param=84*(120+1)=10164

Fc_Gaussian: Param=(84+1)x10=850

  • **(3)lenet5_Codes**
## lenet5\_codes

import os

import glob

import numpy as np

import tensorflow as tf

import matplotlib.pyplot as plt

from skimage import io,transform



#mnist数据集中训练数据和测试数据保存地址

train\_path = "/data/datasets/mnist/train/"

test\_path = "/data/datasets/mnist/test/"

#读取图片及其标签函数

def read\_image(path):

    label\_dir = [path+x  for x in os.listdir(path) if os.path.isdir(path+x)]

    images = []

    labels = []

    for index,folder in enumerate(label\_dir):

        for img in glob.glob(folder+'/\*.png'):

            print("reading the image:%s"%img)

            image = io.imread(img)

            image = transform.resize(image,(w,h,c))

            images.append(image)

            labels.append(index)

    return np.asarray(images,dtype=np.float32),np.asarray(labels,dtype=np.int32)

#读取训练数据及测试数据            

train\_data,train\_label = read\_image(train\_path)

test\_data,test\_label = read\_image(test\_path)

#打乱训练数据及测试数据

train\_image\_index=np.random.shuffle(np.arange(len(train\_data)))

train\_data = train\_data[train\_image\_index]

train\_label = train\_label[train\_image\_index]

#每次获取batch\_size个样本进行训练或测试

def get\_batch(data,label,batch\_size):

    for start\_index in range(0,len(data)-batch\_size+1,batch\_size):

        slice\_index = slice(start\_index,start\_index+batch\_size)

        yield data[slice\_index],label[slice\_index]



# 1.Parameters

train\_epochs = 100

batch\_size = 100

display\_step = 1

learning\_rate = 1e-4

drop\_prob = 0.5

fch\_nodes = 512



# 2.functions

def weight(shape):

    ''' define the initial value of weight

    param : shape=[height,width,input\_channle,output\_channles]

    truncated\_normal() : limite the initial value'voliatility less than two stddev

    '''

    return tf.Variable(tf.truncated\_normal(shape,stddev=0.1,dtype=tf.float32))

def biases(shape):

    ''' define initial value of biases

    param : shape=[None,output\_channles]

    '''

    return tf.Variables(tf.random\_normal(shape,dtype=tf.float32))

def xavier\_init(layer1,layer2,constant=1):

    ''' define the full connection's parameters '''

    Min = -constant/tf.sqrt(6.0/(ayer1+layer2))

    Max =  constant/tf.sqrt(6.0/(ayer1+layer2))

    return tf.variable(tf.random.uniform((ayer1,layer2),minval=Min,maxval=Max,dtype=tf.float32))

def conv2d(x,w):

     ''' converlution '''

     return tf.nn.conv2d(x,w,strides=[1,1,1,1],padding='SAME')

def max\_pool\_2x2(x):

    ''' max pooling '''

    return tf.nn.max\_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')

def cnn(x,train,regularizer):

  

     with tf.variable\_scope('conv1'):

         net1 = tf.nn.relu(tf.nn.bias\_add(conv2d(x,weight([5,5,1,6])),biases(6)))

         net1\_p = max\_pool\_2x2(net1)

     with tf.variable\_scope('conv2'):

         net2 = tf.nn.relu(tf.nn.bias\_add(conv2d(x,weight([5,5,6,16])),biases(16)))

         net2\_p = max\_pool\_2x2(net2)

         

     net2\_p\_shape = net2\_p.get\_shape().as\_list()

     nodes = pool\_shape[1]\*pool\_shape[2]\*pool\_shape[3]

     net2\_p\_reshaped = tf.reshape(net2\_p,[-1,nodes])

     

     with tf.variable\_scope('fc1'):

        if regularizer != None:

            tf.add\_to\_collection('losses',regularizer(weight([None,120])))

        net\_fc1 = tf.nn.relu(tf.matmul(net2\_p\_reshaped,weight([None,120])) + biases(120))

        if train:

            net\_fc1  = tf.nn.dropout(net\_fc1,0.5)

     with tf.variable\_scope('fc2'):

        if regularizer != None:

            tf.add\_to\_collection('losses',regularizer(weight([120,84])))

        net\_fc2 = tf.nn.relu(tf.matmul(net\_fc1,weight([120,84])) + biases(84))

        if train:

            net\_fc2 = tf.nn.dropout(net\_fc2,0.5)

     with tf.variable\_scope('fc3'):

        if regularizer != None:

            tf.add\_to\_collection('losses',regularizer(weight([84,10])))

            y\_p = tf.matmul(net\_fc2,weight([84,10])) + biases(10)

     return y\_p



# 3.搭建CNN

x = tf.placeholder(tf.float32,[None,32,32,1], name='x')

y\_t = tf.placeholder(tf.int32,[None,10],name='y\_t')

y\_p = cnn(x,False,regularizer)

cross\_entropy = tf.reduce\_mean(tf.nn.sparse\_softmax\_cross\_entropy\_with\_logits(logits=y\_P,labels=y\_T))

loss = cross\_entropy + tf.add\_n(tf.get\_collection('losses'))

train\_op = tf.train.AdamOptimizer(0.001).minimize(loss)

accuracy = tf.reduce\_mean(tf.cast(tf.equal(tf.argmax(y\_p,1),tf.argmax(y\_t,1)),tf.float32))       



# 4.创建Session会话

with tf.Session() as sess:

    #初始化所有变量(权值,偏置等)

    sess.run(tf.global\_variables\_initializer())



    #将所有样本训练10次,每次训练中以64个为一组训练完所有样本。

    #train\_num可以设置大一些。

    train\_num = 10

    batch\_size = 64

    for i in range(train\_num):



        train\_loss,train\_acc,batch\_num = 0, 0, 0

        for train\_data\_batch,train\_label\_batch in get\_batch(train\_data,train\_label,batch\_size):

            \_,err,acc = sess.run([train\_op,loss,accuracy],feed\_dict={x:train\_data\_batch,y\_:train\_label\_batch})

            train\_loss+=err;train\_acc+=acc;batch\_num+=1

        print("train loss:",train\_loss/batch\_num)

        print("train acc:",train\_acc/batch\_num)



        test\_loss,test\_acc,batch\_num = 0, 0, 0

        for test\_data\_batch,test\_label\_batch in get\_batch(test\_data,test\_label,batch\_size):

            err,acc = sess.run([loss,accuracy],feed\_dict={x:test\_data\_batch,y\_:test\_label\_batch})

            test\_loss+=err;test\_acc+=acc;batch\_num+=1

        print("test loss:",test\_loss/batch\_num)

        print("test acc:",test\_acc/batch\_num)   

         
2.AlexNet网络结构
屏幕快照 2019-03-17 22.24.58.png
屏幕快照 2019-03-17 22.24.58.png
  • **(1)AlexNet**

conv1->pool1->

conv2->pool2->

conv3_1->conv3_2->conv3_3->

Fc1->Fc2->Fc3

224x224x3(filter=11,strides=4)—>55x55x96(ksize=3,s=2)—>

27x27x96(f=5,s=1)—>27x27x256(k=3,s=2)—>

13x13x256(f=3,s=3)——>13x13x384(f=3,s=3)——>13x13x384(f=3,s=3)——>13x13x256(k=3,s=3)——>

6x6x256—>Fc(4096)——>Fc(4096)——>softmax(1000)

  • **(2)AlexNet_Param**

Input : 224x224x3

conv1 : filter=11,strides=4

feature1: 55x55x96

pool1: ksize=3,s=2

feature:27x27x96

  conv2 : f=5,s=1

feature : 27x27x256

pool2: k=3,s=2

feature :13x13x256

 conv3\_1 : (f=3,s=3)

feature : 13x13x384

conv3_2 : (f=3,s=3)

feature : 13x13x384

conv3_3 : (f=3,s=3)

13x13x256(k=3,s=3)

6x6x256(k=3,s=2)

Fc\_1 (4096)

Fc_2(4096)

softmax(1000)

  • **(3)AlexNet_Codes**
import tensorflow as tf  

import input\_data  

  

mnist = input\_data.read\_data\_sets('MNIST\_data', one\_hot=True)  

  

sess = tf.InteractiveSession()  

  

# 训练数据

x = tf.placeholder("float", shape=[None, 784])  

# 训练标签数据

y\_ = tf.placeholder("float", shape=[None, 10])  

# 把x更改为4维张量,第1维代表样本数量,第2维和第3维代表图像长宽, 第4维代表图像通道数, 1表示黑白

x\_image = tf.reshape(x, [-1, 28, 28, 1])

 

# 第一层:卷积层

# 过滤器大小为5\*5, 当前层深度为1, 过滤器的深度为32

conv1\_weights = tf.get\_variable("conv1\_weights", [5, 5, 1, 32], initializer=tf.truncated\_normal\_initializer(stddev=0.1))

conv1\_biases = tf.get\_variable("conv1\_biases", [32], initializer=tf.constant\_initializer(0.0))

# 移动步长为1, 使用全0填充

conv1 = tf.nn.conv2d(x\_image, conv1\_weights, strides=[1, 1, 1, 1], padding='SAME')

# 激活函数Relu去线性化

relu1 = tf.nn.relu(tf.nn.bias\_add(conv1, conv1\_biases))

  

#第二层:最大池化层  

#池化层过滤器的大小为2\*2, 移动步长为2,使用全0填充  

pool1 = tf.nn.max\_pool(relu1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')  

  

#第三层:卷积层  

conv2\_weights = tf.get\_variable("conv2\_weights", [5, 5, 32, 64], initializer=tf.truncated\_normal\_initializer(stddev=0.1)) #过滤器大小为5\*5, 当前层深度为32, 过滤器的深度为64  

conv2\_biases = tf.get\_variable("conv2\_biases", [64], initializer=tf.constant\_initializer(0.0))  

conv2 = tf.nn.conv2d(pool1, conv2\_weights, strides=[1, 1, 1, 1], padding='SAME') #移动步长为1, 使用全0填充  

relu2 = tf.nn.relu( tf.nn.bias\_add(conv2, conv2\_biases) )  

  

#第四层:最大池化层  

#池化层过滤器的大小为2\*2, 移动步长为2,使用全0填充  

pool2 = tf.nn.max\_pool(relu2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')  

  

#第五层:全连接层  

fc1\_weights = tf.get\_variable("fc1\_weights", [7 \* 7 \* 64, 1024], initializer=tf.truncated\_normal\_initializer(stddev=0.1)) #7\*7\*64=3136把前一层的输出变成特征向量  

fc1\_baises = tf.get\_variable("fc1\_baises", [1024], initializer=tf.constant\_initializer(0.1))  

pool2\_vector = tf.reshape(pool2, [-1, 7 \* 7 \* 64])  

fc1 = tf.nn.relu(tf.matmul(pool2\_vector, fc1\_weights) + fc1\_baises)  

  

#为了减少过拟合,加入Dropout层  

keep\_prob = tf.placeholder(tf.float32)  

fc1\_dropout = tf.nn.dropout(fc1, keep\_prob)  

  

#第六层:全连接层  

fc2\_weights = tf.get\_variable("fc2\_weights", [1024, 10], initializer=tf.truncated\_normal\_initializer(stddev=0.1)) #神经元节点数1024, 分类节点10  

fc2\_biases = tf.get\_variable("fc2\_biases", [10], initializer=tf.constant\_initializer(0.1))  

fc2 = tf.matmul(fc1\_dropout, fc2\_weights) + fc2\_biases  

  

#第七层:输出层  

# softmax  

y\_conv = tf.nn.softmax(fc2)  

  

#定义交叉熵损失函数  

cross\_entropy = tf.reduce\_mean(-tf.reduce\_sum(y\_ \* tf.log(y\_conv), reduction\_indices=[1]))  

  

#选择优化器,并让优化器最小化损失函数/收敛, 反向传播  

train\_step = tf.train.AdamOptimizer(1e-4).minimize(cross\_entropy)  

  

# tf.argmax()返回的是某一维度上其数据最大所在的索引值,在这里即代表预测值和真实值  

# 判断预测值y和真实值y\_中最大数的索引是否一致,y的值为1-10概率  

correct\_prediction = tf.equal(tf.argmax(y\_conv,1), tf.argmax(y\_,1))  

  

# 用平均值来统计测试准确率  

accuracy = tf.reduce\_mean(tf.cast(correct\_prediction, tf.float32))  

  

#开始训练  

sess.run(tf.global\_variables\_initializer())  

for i in range(10000):  

    batch = mnist.train.next\_batch(100)  

    if i%100 == 0:  

        train\_accuracy = accuracy.eval(feed\_dict={x:batch[0], y\_: batch[1], keep\_prob: 1.0}) #评估阶段不使用Dropout  

        print("step %d, training accuracy %g" % (i, train\_accuracy))  

    train\_step.run(feed\_dict={x: batch[0], y\_: batch[1], keep\_prob: 0.5}) #训练阶段使用50%的Dropout  

  

  

#在测试数据上测试准确率  

print("test accuracy %g" % accuracy.eval(feed\_dict={x: mnist.test.images, y\_: mnist.test.labels, keep\_prob: 1.0}))
3.VGGNet网络结构
屏幕快照 2019-03-17 22.25.25.png
屏幕快照 2019-03-17 22.25.25.png
屏幕快照 2019-03-17 22.24.48.png
屏幕快照 2019-03-17 22.24.48.png
  • **(1)VGGNet**

conv1_1->conv1_2->pool1->

conv2_1->conv2_2->pool2->

conv3_1->conv3_2->conv3_3->pool3->

conv4_1->conv4_2->conv4_3->pool4->

conv5_1->conv5_2->conv5_3->pool5->

Fc1->Fc2->Fc3

注释:(filter=3,strides=1);(ksize=1,strides=2)

224x224x3(f=3,s=1)—>224x224x64(f=3,s=1)——>224x224x64(k=2,s=2)—>

112x112x128(f=3,s=1)—>112x112x128(f=3,s=1)—>112x112x128(f=3,s=1)—>

56x56x128(f=3,s=1)—>56x56x256(f=3,s=1)—>56x56x256(f=3,s=1)—>56x56x256(k=2,s=2)—>28x28x256(f=3,s=1)—>28x28x512(f=3,s=1)—>28x28x512(f=3,s=1)—>28x28x512(k=2,s=2)—>14x14x512(f=3,s=1)—>14x14x512(f=3,s=1)—>14x14x512(f=3,s=1)—>14x14x512(k=2,s=2)—>7x7x512—>Fc(9216)—>Fc(4096)—>softmax(1000)

  • **(2)VGGNet_Param**

input:224x224x3

conv1_1 : (f=3,s=1)

feature : 224x224x64

conv1_2 : (f=3,s=1)

feature: 224x224x64

pool1 : (k=2,s=2)

feature : 112x112x128

conv2\_1 : (f=3,s=1)

feature : 112x112x128

conv2_2 : (f=3,s=1)

feature : 112x112x128

pool2 : (k=2,s=2)

feature : 56x56x256

conv3\_1 : (f=3,s=1)

feature : 56x56x256

conv3_2 : (f=3,s=1)

feature : 56x56x256

conv3_3 : (f=3,s=1)

feature : 56x56x256

pool3 : (k=2,s=2)

28x28x256

conv4\_1 : (f=3,s=1)

feature : 28x28x512

conv4_2 : (f=3,s=1)

feature : 28x28x512

conv4_3 :(f=3,s=1)

feature : 28x28x512

pool4 :(k=2,s=2)

feature : 14x14x512

conv4\_1 :(f=3,s=1)

feature : 14x14x512

conv4_2 :(f=3,s=1)

feature : 14x14x512

conv4_3 :(f=3,s=1)

feature : 14x14x512

pool4 :(k=2,s=2)

feature : 7x7x512

Fc(9216)—>Fc(4096)—>softmax(1000)
  • **(3)VGGNet_Codes**
import tensorflow as tf

import numpy as np

# VGG16 output class number 1000 -> weights: 137,557,696

def max\_pool(x, name):

    return tf.layers.max\_pooling2d(x, pool\_size=2, strides=2, padding='same', name=name)

def dropout(x, keepPro, name=None):

    return tf.nn.dropout(x, keepPro, name)

def conv\_layer(x, f, size, name):

     with tf.variable\_scope(name):

    # contain kernal and bias

        conv = tf.layers.conv2d(x, f, kernel\_size=size, padding='same', name=name)

        relu = tf.nn.relu(conv)

    return relu

def fc\_layer(x, inputD, outputD, name, relu=False):

    with tf.variable\_scope(name):

        w = tf.get\_variable('w', shape=[inputD, outputD], dtype='float32')

        b = tf.get\_variable('b', shape=[outputD], dtype='float32')

        out = tf.nn.xw\_plus\_b(x, w, b, name=name)

if relu:

            out = tf.nn.relu(out)

    return out

class Vgg():

    def \_\_init\_\_(self, im, class\_num, isvgg19, modelpath='./models/vgg16.npy'):

        self.input\_x = im

        self.class\_num = class\_num

        self.vgg19 = isvgg19

        self.modelpath = modelpath

    def build(self, isTraining=True):

# 224x224x3 -> 224x224x64 -> 224x224x64 -> 112x112x64

        conv1\_1 = conv\_layer(self.input\_x, 64, 3, name='conv1\_1')

        conv1\_2 = conv\_layer(conv1\_1, 64, 3, name='conv1\_2')

        pool1 = max\_pool(conv1\_2, name='pool1')

# 112x112x64 -> 112x112x128 -> 112x112x128 -> 56x56x128

        conv2\_1 = conv\_layer(pool1, 128, 3, name='conv2\_1')

        conv2\_2 = conv\_layer(conv2\_1, 128, 3, name='conv2\_2')

        pool2 = max\_pool(conv2\_2, name='pool2')

# 56x56x128 -> 56x56x256 -> 56x56x256 -> 56x56x256 -> 28x28x256

        conv3\_1 = conv\_layer(pool2, 256, 3, name='conv3\_1')

        conv3\_2 = conv\_layer(conv3\_1, 256, 3, name='conv3\_2')

        conv3\_3 = conv\_layer(conv3\_2, 256, 3, name='conv3\_3')

        if self.vgg19:

            conv3\_4 = conv\_layer(conv3\_3, 256, 3, name='conv3\_4')

            pool3 = max\_pool(conv3\_4, name='pool3')

       else:

            pool3 = max\_pool(conv3\_3, name='pool3')

# 28x28x256 -> 28x28x512 -> 28x28x512 -> 28x28x512 -> 14x14x512

        conv4\_1 = conv\_layer(pool3, 512, 3, name='conv4\_1')

        conv4\_2 = conv\_layer(conv4\_1, 512, 3, name='conv4\_2')

        conv4\_3 = conv\_layer(conv4\_2, 512, 3, name='conv4\_3')

       if self.vgg19:

            conv4\_4 = conv\_layer(conv4\_3, 512, 3, name='conv4\_4')

            pool4 = max\_pool(conv4\_4, name='pool4')

       else:

            pool4 = max\_pool(conv4\_3, name='pool4')

# 14x14x512 -> 14x14x512 -> 14x14x512 ->14x14x512 -> 7x7x512

               conv5\_1 = conv\_layer(pool4, 512, 3, name='conv5\_1')

               conv5\_2 = conv\_layer(conv5\_1, 512, 3, name='conv5\_2')

              conv5\_3 = conv\_layer(conv5\_2, 512, 3, name='conv5\_3')

       if self.vgg19:

            conv5\_4 = conv\_layer(conv5\_3, 512, 3, name='conv5\_4')

            pool5 = max\_pool(conv5\_4, name='pool5')

       else:

            pool5 = max\_pool(conv5\_3, name='pool5')

               fc\_in = tf.reshape(pool5, [-1, 7\*7\*512])

# 7\*7\*512 -> 4096

              fc6 = fc\_layer(fc\_in, 7\*7\*512, 4096, name='fc6', relu=True)

        if isTraining:

                  fc6 = dropout(fc6, 0.5)

# 4096 -> 4096

        fc7 = fc\_layer(fc6, 4096, 4096, name='fc7', relu=True)

        if isTraining:

        fc7 = dropout(fc7, 0.5)

# 4096 -> class numbe

        fc8 = fc\_layer(fc7, 4096, self.class\_num, name='fc8', relu=False)

        prob = tf.nn.softmax(fc8, name='prob')

        return prob, fc8

def loadModel(self, sess, isFineTuring=False):

        wData = np.load(self.modelpath, encoding='bytes').item()

                for name in wData:

                    with tf.variable\_scope(name, reuse=True):

                       for p in wData[name]:

                           if len(p.shape) == 1:

                          # bias

                           if name.startswith('conv'):

                            sess.run(tf.get\_variable(name+'/bias', trainable=False).assign(p))

                           elif (not name.startswith('fc8')) or (not isFineTuring):

                            sess.run(tf.get\_variable('b', trainable=False).assign(p))

                           else:

                            # weights

                          if name.startswith('conv'):

                            sess.run(tf.get\_variable(name+'/kernel', trainable=False).assign(p))

                           elif (not name.startswith('fc8')) or (not isFineTuring):

                            sess.run(tf.get\_variable('w', trainable=False).assign(p))

def losses(self, labels, logits):

        loss = tf.reduce\_mean(tf.nn.softmax\_cross\_entropy\_with\_logits(labels=labels, logits=logits))

#loss = tf.nn.softmax\_cross\_entropy\_with\_logits(labels=labels, logits=logits)

     return loss

def accurracy(self, labels, logits):

        prediction = tf.to\_int64(tf.argmax(logits, 1))

        correct\_prediction = tf.equal(prediction, tf.argmax(labels,1))

        accurracy = tf.reduce\_mean(tf.cast(correct\_prediction, tf.float32))

         return accurracy
4.InterceptNet网络结构
屏幕快照 2019-03-17 22.25.10.png
屏幕快照 2019-03-17 22.25.10.png

Inception Module的基本结构,其中有4个分支:第一个分支对输入进行1´1的卷积,这其实也是NIN中提出的一个重要结构。1´1的卷积是一个非常优秀的结构,它可以跨通道组织信息,提高网络的表达能力,同时可以对输出通道升维和降维。可以看到Inception Module的4个分支都用到了1´1卷积,来进行低成本(计算量比3´3小很多)的跨通道的特征变换。第二个分支先使用了1´1卷积,然后连接3´3卷积,相当于进行了两次特征变换。第三个分支类似,先是1´1的卷积,然后连接5´5卷积。最后一个分支则是3´3最大池化后直接使用1´1卷积。我们可以发现,有的分支只使用1´1卷积,有的分支使用了其他尺寸的卷积时也会再使用1´1卷积,这是因为1´1卷积的性价比很高,用很小的计算量就能增加一层特征变换和非线性化。Inception Module的4个分支在最后通过一个聚合操作合并(在输出通道数这个维度上聚合)。Inception Module中包含了3种不同尺寸的卷积和1个最大池化,增加了网络对不同尺度的适应性,这一部分和Multi-Scale的思想类似。早期计算机视觉的研究中,受灵长类神经视觉系统的启发,Serre使用不同尺寸的Gabor滤波器处理不同尺寸的图片,Inception V1借鉴了这种思想。Inception V1的论文中指出,Inception Module可以让网络的深度和宽度高效率地扩充,提升准确率且不致于过拟合。

下载.png
下载.png
  • **(1)Inception_Codes**
import tensorflow as tf
5.ResNet网络结构

假定某段神经网络的输入是x,期望输出是,如果我们直接把输入x传到输出作为初始结果,那么此时我们需要学习的目标就是。如图14所示,这就是一个ResNet的残差学习单元(Residual Unit),ResNet相当于将学习目标改变了,不再是学习一个完整的输出,只是输出和输入的差别,即残差。

在ResNet的论文中,除了提出图14中的两层残差学习单元,还有三层的残差学习单元。两层的残差学习单元中包含两个相同输出通道数(因为残差等于目标输出减去输入,即,因此输入、输出维度需保持一致)的3´3卷积;而3层的残差网络则使用了Network In Network和Inception Net中的1´1卷积,并且是在中间3´3的卷积前后都使用了1´1卷积,有先降维再升维的操作。另外,如果有输入、输出维度不同的情况,我们可以对x做一个线性映射变换维度,再连接到后面的层

屏幕快照 2019-03-17 22.25.18.png
屏幕快照 2019-03-17 22.25.18.png

下图是VGGNet-19,以及一个34层深的普通卷积网络,和34层深的ResNet网络的对比图。可以看到普通直连的卷积神经网络和ResNet的最大区别在于,ResNet有很多旁路的支线将输入直接连到后面的层,使得后面的层可以直接学习残差,这种结构也被称为shortcut或skip connections。

屏幕快照 2019-03-17 22.25.25.png
屏幕快照 2019-03-17 22.25.25.png
  • **(1)ResNet_Codes**
import tensorflow as tf

参考:https://www.leiphone.com/news/201702/dgpHuriVJHTPqqtT.html

https://blog.csdn.net/enchanted_zhouh/article/details/76855108

https://blog.csdn.net/OliverkingLi/article/details/73849228

https://blog.csdn.net/stesha_chen/article/details/81197388

h.com/stesha2016/tensorflow-classification/blob/master/networks/vgg.pyttps://github

https://zhuanlan.zhihu.com/p/34055904

https://cloud.tencent.com/developer/article/1010309

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 目录
    • 1.lenet5网络结构
      • 2.AlexNet网络结构
        • 3.VGGNet网络结构
          • 4.InterceptNet网络结构
            • 5.ResNet网络结构
            领券
            问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档