Caffe源码调试

这篇文件主要介绍如何使用Linux的gdb调试Caffe的源码,源码调试主要是为了阅读并更好的了解Caffe源码。

1. 准备工作

  1. 首先要在编译Caffe源码时打开debug模式,即将Makefile.config中的DEBUG := 1打开。
  2. 下载mnist数据集,主要是在mnist数据集上进行调试,执行bash data/mnist/get_mnist.sh
  3. 转换mnist数据集为LMDB,bash examples/mnist/create_mnist.sh
  4. 修改examples/mnist/lenet_solver.prototxt,将GPU改为CPU。

2. 调试

1. 激活GDB

使用GDB启动调试,执行gdb --args build/tools/caffe train --solver examples/mnist/lenet_solver.prototxt--args表示我们调试时需要输入的参数,调试的命令为build/tools/caffe,caffe命令的参数为--solver examples/mnist/lenet_solver.prototxt

执行结果:

$ gdb --args build/tools/caffe train --solver examples/mnist/lenet_solver.prototxt
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/irteam/line-brain/deploy/caffe/.build_debug/tools/caffe.bin...done.

2. 设置断点

执行b src/caffe/layers/base_conv_layer.cpp:117b表示插入断点(breakpoint),断点的位置是base_conv_layer.cpp文件中的117行。插入断点的命令形式为:

b path/to/code.cpp:#line

118行相关代码:

117 channels_ = bottom[0]->shape(channel_axis_);
118 num_output_ = this->layer_param_.convolution_param().num_output();
119 CHECK_GT(num_output_, 0);

执行结果:

(gdb) b src/caffe/layers/base_conv_layer.cpp:117
No source file named src/caffe/layers/base_conv_layer.cpp.
Make breakpoint pending on future shared library load? (y or [n]) y

Breakpoint 1 (src/caffe/layers/base_conv_layer.cpp:117) pending.

3. 运行程序

运行程序的命令是r

执行结果:

Starting program: /*/caffe/build/tools/caffe train --solver examples/mnist/lenet_solver.prototxt
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
I0718 15:19:19.671941 29986 caffe.cpp:211] Use CPU.
[New Thread 0x7fffd81c7700 (LWP 29991)]
[New Thread 0x7fffd79c6700 (LWP 29992)]
I0718 15:19:20.437239 29986 solver.cpp:44] Initializing solver from parameters:
test_iter: 100
test_interval: 500
base_lr: 0.01
display: 100
max_iter: 10000
lr_policy: "inv"
gamma: 0.0001
power: 0.75
momentum: 0.9
weight_decay: 0.0005
snapshot: 5000
snapshot_prefix: "examples/mnist/lenet"
solver_mode: CPU
net: "examples/mnist/lenet_train_test.prototxt"
train_state {
  level: 0
  stage: ""
}
I0718 15:19:20.437687 29986 solver.cpp:87] Creating training net from net file: examples/mnist/lenet_train_test.prototxt
I0718 15:19:20.438357 29986 net.cpp:294] The NetState phase (0) differed from the phase (1) specified by a rule in layer mnist
I0718 15:19:20.438398 29986 net.cpp:294] The NetState phase (0) differed from the phase (1) specified by a rule in layer accuracy
I0718 15:19:20.438499 29986 net.cpp:51] Initializing net from parameters:
name: "LeNet"
state {
  phase: TRAIN
  level: 0
  stage: ""
}
layer {
  name: "mnist"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    scale: 0.00390625
  }
  data_param {
    source: "examples/mnist/mnist_train_lmdb"
    batch_size: 64
    backend: LMDB
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 20
    kernel_size: 5
    stride: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "pool1"
  top: "conv2"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  convolution_param {
    num_output: 50
    kernel_size: 5
    stride: 1
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 2
    stride: 2
  }
}
layer {
  name: "ip1"
  type: "InnerProduct"
  bottom: "pool2"
  top: "ip1"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 500
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "ip1"
  top: "ip1"
}
layer {
  name: "ip2"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  param {
    lr_mult: 1
  }
  param {
    lr_mult: 2
  }
  inner_product_param {
    num_output: 10
    weight_filler {
      type: "xavier"
    }
    bias_filler {
      type: "constant"
    }
  }
}
layer {
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "ip2"
  bottom: "label"
  top: "loss"
}
I0718 15:19:20.439380 29986 layer_factory.hpp:77] Creating layer mnist
I0718 15:19:20.439625 29986 db_lmdb.cpp:35] Opened lmdb examples/mnist/mnist_train_lmdb
I0718 15:19:20.439702 29986 net.cpp:84] Creating Layer mnist
I0718 15:19:20.439735 29986 net.cpp:380] mnist -> data
I0718 15:19:20.439853 29986 net.cpp:380] mnist -> label
I0718 15:19:20.444980 29986 data_layer.cpp:45] output data size: 64,1,28,28
I0718 15:19:20.445436 29986 base_data_layer.cpp:72] Initializing prefetch
[New Thread 0x7fffd603d700 (LWP 29993)]
I0718 15:19:20.448151 29986 base_data_layer.cpp:75] Prefetch initialized.
I0718 15:19:20.448186 29986 net.cpp:122] Setting up mnist
I0718 15:19:20.448216 29986 net.cpp:129] Top shape: 64 1 28 28 (50176)
I0718 15:19:20.448235 29986 net.cpp:129] Top shape: 64 (64)
I0718 15:19:20.448245 29986 net.cpp:137] Memory required for data: 200960
I0718 15:19:20.448264 29986 layer_factory.hpp:77] Creating layer conv1
I0718 15:19:20.448324 29986 net.cpp:84] Creating Layer conv1
I0718 15:19:20.448345 29986 net.cpp:406] conv1 <- data
I0718 15:19:20.448393 29986 net.cpp:380] conv1 -> conv1

Breakpoint 1, caffe::BaseConvolutionLayer<float>::LayerSetUp (this=0x91edd70,
    bottom=std::vector of length 1, capacity 1 = {...}, top=std::vector of length 1, capacity 1 = {...})
    at src/caffe/layers/base_conv_layer.cpp:117
117       channels_ = bottom[0]->shape(channel_axis_);
Missing separate debuginfos, use: debuginfo-install OpenEXR-libs-1.7.1-7.el7.x86_64 atk-2.14.0-1.el7.x86_64 atlas-3.10.1-10.el7.x86_64 boost-filesystem-1.53.0-26.el7.x86_64 boost-python-1.53.0-26.el7.x86_64 boost-system-1.53.0-26.el7.x86_64 boost-thread-1.53.0-26.el7.x86_64 cairo-1.14.2-1.el7.x86_64 expat-2.1.0-10.el7_3.x86_64 fontconfig-2.10.95-10.el7.x86_64 freetype-2.4.11-12.el7.x86_64 gdk-pixbuf2-2.31.6-3.el7.x86_64 gflags-2.1.1-6.el7.x86_64 glib2-2.46.2-4.el7.x86_64 glibc-2.17-157.el7_3.1.x86_64 glog-0.3.3-8.el7.x86_64 graphite2-1.3.6-1.el7_2.x86_64 gstreamer-0.10.36-7.el7.x86_64 gstreamer-plugins-base-0.10.36-10.el7.x86_64 gtk2-2.24.28-8.el7.x86_64 harfbuzz-0.9.36-1.el7.x86_64 hdf5-1.8.12-8.el7.x86_64 ilmbase-1.0.3-7.el7.x86_64 jasper-libs-1.900.1-29.el7.x86_64 jbigkit-libs-2.0-11.el7.x86_64 leveldb-1.12.0-11.el7.x86_64 libX11-1.6.3-3.el7.x86_64 libXau-1.0.8-2.1.el7.x86_64 libXcomposite-0.4.4-4.1.el7.x86_64 libXcursor-1.1.14-2.1.el7.x86_64 libXdamage-1.1.4-4.1.el7.x86_64 libXext-1.3.3-3.el7.x86_64 libXfixes-5.0.1-2.1.el7.x86_64 libXi-1.7.4-2.el7.x86_64 libXinerama-1.1.3-2.1.el7.x86_64 libXrandr-1.4.2-2.el7.x86_64 libXrender-0.9.8-2.1.el7.x86_64 libffi-3.0.13-18.el7.x86_64 libgcc-4.8.5-11.el7.x86_64 libgfortran-4.8.5-11.el7.x86_64 libjpeg-turbo-1.2.90-5.el7.x86_64 libpng-1.5.13-7.el7_2.x86_64 libquadmath-4.8.5-11.el7.x86_64 libselinux-2.5-6.el7.x86_64 libstdc++-4.8.5-11.el7.x86_64 libtiff-4.0.3-27.el7_3.x86_64 libv4l-0.9.5-4.el7.x86_64 libxcb-1.11-4.el7.x86_64 libxml2-2.9.1-6.el7_2.3.x86_64 lmdb-libs-0.9.18-1.el7.x86_64 opencv-2.4.5-3.el7.x86_64 opencv-core-2.4.5-3.el7.x86_64 orc-0.4.22-5.el7.x86_64 pango-1.36.8-2.el7.x86_64 pcre-8.32-15.el7_2.1.x86_64 pixman-0.34.0-1.el7.x86_64 protobuf-2.5.0-8.el7.x86_64 python-libs-2.7.5-48.el7.x86_64 snappy-1.1.0-3.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64

Breakpoint 1之前是正常的程序日志输出,程序在断点处暂停。

查看变量命令为p var,命令与结果如下:

(gdb) p channels_
$1 = 0

(gdb) p channel_axis_
$2 = 1

此时,channels_值为0。下一行命令为n,执行结果如下:

(gdb) n
118       num_output_ = this->layer_param_.convolution_param().num_output();

此时查看channels_值为1,mnist数据是灰度图像,channels_1没问题:

(gdb) p channels_
$3 = 1

命令c是继续执行直到下一个断点。

如果需要调试GPU程序,可以使用cuda-gdb,文档地址为:http://docs.nvidia.com/cuda/cuda-gdb/index.html#axzz4nAAR7ujZ

参考资料

  1. http://zhaok.xyz/blog/post/debug-caffe/

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏顶级程序员

干货 | Python 爬虫的工具列表大全

源 / 伯乐头条 这个列表包含与网页抓取和数据处理的Python库。 网络 通用 urllib -网络库(stdlib)。 requests -网络库。 gr...

2996
来自专栏Python数据科学

要成为一个专业的爬虫大佬,你还需要了解这些

本文内容参考Github:https://github.com/lorien/awesome-web-scraping/blob/master/python.m...

651
来自专栏Linux Python 加油站

Python学习干货 史上最全的 Python 爬虫工具列表大全

链接:https://mp.weixin.qq.com/s/UkXT20Oko6oYbeo7zavCNA

1002
来自专栏技术专栏

scrapy进阶开发(一):scrapy架构源码分析

其子类有HtmlResponse,TextResponse,XmlResponse

943
来自专栏Java进阶架构师

dubbo源码解析-详解router

估算了一下,dubbo里面涉及的东西还是比较多的.比如谈到框架的时候,设计模式都是一个老生常谈的话题,再比如我们开发中我们不常用的一些概念,spi、javass...

573
来自专栏沈唁志

【收藏】Python 爬虫的工具列表大全

1744
来自专栏机器学习算法与Python学习

干货 | Python 爬虫的工具列表大全

源 | 伯乐头条 | 小象 这个列表包含与网页抓取和数据处理的Python库。 网络 通用 urllib -网络库(stdlib)。 requests -网络...

2869
来自专栏芋道源码1024

Dubbo源码解析 —— Router

前言 估算了一下, dubbo里面涉及的东西还是比较多的.比如谈到框架的时候,设计模式都是一个老生常谈的话题,再比如我们开发中我们不常用的一些概念, spi、...

3826
来自专栏高性能服务器开发

+从零实现一款12306刷票软件1.4

这里还有个注意细节,就是通过POST请求发送的数据需要对一些符号做URL Encode,这个我在上一篇文章《从零实现一个http服务器》也详细做了介绍,还不清楚...

1352
来自专栏Aloys的开发之路

Python第三方常用工具、库、框架等

       Python ImagingLibrary(PIL):它提供强大的图形处理的能力,并提供广泛的图形文件格式支持,该库能进行图形格式的转换、打印和显...

33210

扫码关注云+社区