Deeplab v2 安装及调试全过程

上期为大家带来的是从FCN到DeepLab V2的一些相关知识，今天我们就来和大家分享一些DeepLab V2的安装及调试全过程，希望可以为一些需要的科研小伙伴带来一丝丝帮助，请继续欣赏下去。把Deeplabv2的 run_pascal.sh与run_densecrf.sh成功运行，现将调试过程整理如下：

一、安装必要的依赖库

安装 matio：

安装方法1： sudo apt-get install libmatio-dev

安装方法2： 下载matio (https://sourceforge.net/projects/matio/files/matio/1.5.2/)

tar zxf matio-1.5.2.tar.gz

cd matio-1.5.2

./configure

make

make check

make install

sudo ldconfig

安装 wget

sudo pip install wget

如果出错，就按照下面的命令成功：

pip install –upgrade pip –user

pip install –upgrade setuptools –user

sudo pip install wget

二、下载Deeplabv2并编译

下载代码：

git clone https://github.com/xmojiao/deeplab_v2.git

（试过许多Deeplab代码，这个最容易编译成功，所以我用的是这个代码编译的）

对 caffe 进行编译：

修改deeplab_v2/deeplab-public-ver2/路径下的Makefile.config.example文件，重命名为Makefile.config；

接着修改这个文件中的内容，将第四行的 “# USE_CUDNN := 1”的 # 去掉。如果需要，因为我用的pycaffe编译，所以不需要修改python的路径，保存退出。

编译 caffe的命令：

cd ~/Desktop/deeplab_v2/deeplab-public-ver2

make all -j16

如果出现下面的错误1：

src/caffe/net.cpp:8:18: fatal error: hdf5.h: No such file or directory compilation terminated.

解决办法: 修改两个make文件（Makefile.config，Makefile）

Makefile.config：

将

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

修改为：

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnumake

Makefile：

将

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5

修改为：

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial matio

重新编译：

make all -j16

如果出现下面的错误2：

./include/caffe/common.cuh(9): error: function “atomicAdd(double *, double)” has already been defined

解决方法：

打开./include/caffe/common.cuh文件，在atomicAdd前添加宏判断即可。 下面为修改后文件：

// Copyright 2014 George Papandreou#ifndef CAFFE_COMMON_CUH_#defineCAFFE_COMMON_CUH_#include // CUDA: atomicAdd is not defined for doubles#if!defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 600#elsestatic__inline__ __device__doubleatomicAdd(double*address,doubleval) { unsignedlonglongint* address_as_ull = (unsignedlonglongint*)address; unsignedlonglongintold = *address_as_ull, assumed;if(val==0.0)return__longlong_as_double(old);do{ assumed = old; old = atomicCAS(address_as_ull, assumed, __double_as_longlong(val +__longlong_as_double(assumed))); }while(assumed != old);return__longlong_as_double(old); }#endif#endif

继续编译：

make all -j16

如果出现下面的错误3：

:.build_release/lib/libcaffe.so:undefined reference to `cudnnConvolutionBackwardFilter_v3’

解决方法：

将BVLC(https://github.com/BVLC/caffe)中的下列文件copy 到相应的文件夹：

./include/caffe/util/cudnn.hpp ./include/caffe/layers/cudnn_conv_layer.hpp ./include/caffe/layers/cudnn_relu_layer.hpp ./include/caffe/layers/cudnn_sigmoid_layer.hpp ./include/caffe/layers/cudnn_tanh_layer.hpp ./src/caffe/layers/cudnn_conv_layer.cpp ./src/caffe/layers/cudnn_conv_layer.cu ./src/caffe/layers/cudnn_relu_layer.cpp ./src/caffe/layers/cudnn_relu_layer.cu ./src/caffe/layers/cudnn_sigmoid_layer.cpp ./src/caffe/layers/cudnn_sigmoid_layer.cu ./src/caffe/layers/cudnn_tanh_layer.cpp ./src/caffe/layers/cudnn_tanh_layer.cu

然后：

make clean

make all -j16

make pycaffe -j16

这个时候一般都是编译成功。

三、对 run_pascal.sh 进行调试：

首先准备好数据 ：

（参考：http://blog.csdn.net/Xmo_jiao/article/details/77897109）

cd ~/Desktop

mkdir -p my_dataset

# augmented PASCAL VOC

cd my_dataset/

wget http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/semantic_contours/benchmark.tgz

# 1.3 GB

tar -zxvf benchmark.tgz

mv benchmark_RELEASE VOC_aug

# original PASCAL VOC 2012

wget http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar

# 2 GB

tar -xvf VOCtrainval_11-May-2012.tar

mv VOCdevkit/VOC2012 VOC2012_orig && rm -r VOCdevkit

数据转换

因为pascal voc2012增强数据集的label是mat格式的文件，要把mat格式的label转为png格式的图片

~/Desktop/my_dataset/VOC_aug/dataset

mkdir cls_png

cd ~/Desktop/deeplab_v2/voc2012/

./mat2png.py ~/Desktop/my_dataset/VOC_aug/dataset/cls/Desktop/my_dataset/VOC_aug/dataset/cls_png

因为pascal voc2012原始数据集的label为三通道RGB图像，但是caffe最后一层softmax loss层只能识别一通道的label，所以此处我们需要对原始数据集的label进行降维

cd ~/Desktop/my_dataset/VOC2012_orig mkdir SegmentationClass_1D

cd ~/Desktop/deeplab_v2/voc2012/

./convert_labels.py ~/Desktop/my_dataset/VOC2012_orig/SegmentationClass/ ~/Desktop/my_dataset /VOC2012_orig/ImageSets/Segmentation/trainval.txt ~/Desktop/my_dataset/VOC2012_orig/Segmentat ionClass_1D/

数据融合

此时已经处理好好pascal voc2012 增强数据集和pascal voc2012的原始数据集，为了便于train.txt等文件的调用，将两个文件夹数据合并到同一个文件中.现有文件目录如下：

现分别pascal voc2012增强数据集里的images和labels复制到增强数据集中，若重复则覆盖，合将并数据集的操作如下：

cp ~/Desktop/my_dataset/VOC2012_orig/SegmentationClass_1D/* ~/Desktop/my_dataset/VOC_aug/dataset/cls_png

cp ~/Desktop/my_dataset/VOC2012_orig/JPEGImages/* ~/Desktop/my_dataset/VOC_aug/dataset/img/

文件名修改

对应train.txt文件的数据集文件名，修改文件名。

cd ~/Desktop/my_dataset/VOC_aug/dataset

mv ./img ./JPEGImages

那么我们这个阶段使用的数据已经整理完成

四、修改并运行 run_pascal.sh

准备必要的文件 需要的文件从这里下载 deeplabv2 有两种模型（vgg，Res102），vgg ，http://liangchiehchen.com/projects/DeepLab_Models.html

总共需要的文件如图所示：

下载的代码中Desktop/deeplab_v2/voc2012/list已经有了list文件，所以不用重新下载。

/Desktop/deeplab_v2/voc2012/config/deeplab_largeFOV中也有了相应的文件，所以也无需下载。

Desktop/deeplab_v2/voc2012/model/deeplab_largeFOV里没有model，需要把下载好的model放入文件，如图所示：

至此，所有需要的文件全部完毕。

五、运行 train 和 test

进入/Desktop/deeplab_v2/voc2012，修改run_pascal.sh文件，主要是修改路径，我的修改后的文件如下：

#!/bin/sh## MODIFY PATH for YOUR SETTINGROOT_DIR=/home/mmt/Desktop/my_datasetCAFFE_DIR=/home/mmt/Desktop/deeplab_v2/deeplab-public-ver2CAFFE_BIN=$/build/tools/caffe.binEXP=.if["$"="."];thenNUM_LABELS=21DATA_ROOT=$/VOC_aug/dataset/elseNUM_LABELS=echo"Wrong exp name"fi## Specify which model to train########### voc12 ################NET_ID=deeplab_largeFOV## Variables used for weakly or semi-supervisedly training#TRAIN_SET_SUFFIX=TRAIN_SET_SUFFIX=_aug#TRAIN_SET_STRONG=train#TRAIN_SET_STRONG=train200#TRAIN_SET_STRONG=train500#TRAIN_SET_STRONG=train1000#TRAIN_SET_STRONG=train750#TRAIN_SET_WEAK_LEN=5000DEV_ID=####### Create dirsCONFIG_DIR=$/config/$MODEL_DIR=$/model/$mkdir -p$LOG_DIR=$/log/$mkdir -p$exportGLOG_log_dir=$## RunRUN_TRAIN=1#1时trainRUN_TEST=#1时testRUN_TRAIN2=RUN_TEST2=## Training #1 (on train_aug)if[$-eq1];then#LIST_DIR=$/list TRAIN_SET=train$if[ -z$];thenTRAIN_SET_WEAK=$_diff_$comm -3$/$.txt$/$.txt >$/$.txtelseTRAIN_SET_WEAK=$_diff_$_head$comm -3$/$.txt$/$.txt | head -n$>$/$.txtfi#MODEL=$/model/$/init.caffemodel#echoTraining net$/$forpnameintrain solver;dosed"$(eval echo $(cat sub.sed))"\$/$.prototxt >$/$_$.prototxtdoneCMD="$train \ --solver=$/solver_$.prototxt \ --gpu=$"if[-f$];thenCMD="$--weights=$"fiechoRunning$&&$fi## Test #1 specification (on val or test)if[$-eq1];then#forTEST_SETinval;doTEST_ITER=`cat$/list/$.txt | wc-l` MODEL=$/model/$/test.caffemodelif[ !-f$];thenMODEL=`ls -t$/model/$/train_iter_*.caffemodel | head -n1`fi#echoTesting net$/$FEATURE_DIR=$/features/$mkdir -p$/$/fc8 mkdir -p$/$/fc9 mkdir -p$/$/seg_score sed"$(eval echo $(cat sub.sed))"\$/test.prototxt >$/test_$.prototxt CMD="$test \ --model=$/test_$.prototxt \ --weights=$\ --gpu=$\ --iterations=$"echoRunning$&&$donefi## Training #2 (finetune on trainval_aug)if[$-eq1];then#LIST_DIR=$/list TRAIN_SET=trainval$if[ -z$];thenTRAIN_SET_WEAK=$_diff_$comm -3$/$.txt$/$.txt >$/$.txtelseTRAIN_SET_WEAK=$_diff_$_head$comm -3$/$.txt$/$.txt | head -n$>$/$.txtfi#MODEL=$/model/$/init2.caffemodelif[ !-f$];thenMODEL=`ls -t$/model/$/train_iter_*.caffemodel | head -n1`fi#echoTraining2 net$/$forpnameintrain solver2;dosed"$(eval echo $(cat sub.sed))"\$/$.prototxt >$/$_$.prototxtdoneCMD="$train \ --solver=$/solver2_$.prototxt \ --weights=$\ --gpu=$"echoRunning$&&$fi## Test #2 on official test setif[$-eq1];then#forTEST_SETinval test;doTEST_ITER=`cat$/list/$.txt | wc-l` MODEL=$/model/$/test2.caffemodelif[ !-f$];thenMODEL=`ls -t$/model/$/train2_iter_*.caffemodel | head -n1`fi#echoTesting2 net$/$FEATURE_DIR=$/features2/$mkdir -p$/$/fc8 mkdir -p$/$/crf sed"$(eval echo $(cat sub.sed))"\$/test.prototxt >$/test_$.prototxt CMD="$test \ --model=$/test_$.prototxt \ --weights=$\ --gpu=$\ --iterations=$"echoRunning$&&$donefi

接下来运行代码：

Train：

~/Desktop/deeplab_v2/voc2012

sh ./run_pascal.sh

运行结果如下：

Test：

将相应变量改为1：

~/Desktop/deeplab_v2/voc2012

sh ./run_pascal.sh

运行结果如下：

因为结果保存的是mat文件，如果想转换成png的形式，运行：

cd ~/Desktop/deeplab_v2/voc2012

修改create_labels_21.py的路径，然后此目录运行：

python create_labels_21.py

六、修改并运行 run_densecrf.sh

首先对densecrf进行编译。

cd ~/Desktop/deeplab_v2/deeplab-public-ver2/densecrf/ make

有许多warning，但是没出错，不用管。

数据整理

因为densecrf只识别ppm格式的图像，所以要转换格式。

进入/Desktop/deeplab_v2/deeplab-public-ver2/densecrf/my_script，里面有自带的修改ppm 的MATLAB程序，修改路径，直接运行即可。

代码如下：

% save jpg images as bin file for cpp%is_server =1;dataset ='voc2012';%'coco', 'voc2012'ifis_serverifstrcmp(dataset,'voc2012') img_folder ='/home/mmt/Desktop/my_dataset/VOC_aug/dataset/JPEGImages'save_folder ='/home/mmt/Desktop/my_dataset/VOC_aug/dataset/PPMImages';elseifstrcmp(dataset,'coco') img_folder ='/rmt/data/coco/JPEGImages'; save_folder ='/rmt/data/coco/PPMImages';endelseimg_folder ='../img'; save_folder ='../img_ppm';endif~exist(save_folder,'dir') mkdir(save_folder);endimg_dir = dir(fullfile(img_folder,'*.jpg'));fori=1:numel(img_dir) fprintf(1,'processing %d (%d)...\n',i,numel(img_dir)); img = imread(fullfile(img_folder, img_dir(i).name)); img_fn = img_dir(i).name(1:end-4); save_fn = fullfile(save_folder,[img_fn,'.ppm']); imwrite(img, save_fn);end

接下来，修改 run_densecrf.sh， 注意把MODEL_NAME=deeplab_largeFOV修改了。

DATASET=voc2012 修改；SAVE_DIR=/home/mmt/Desktop/deeplab_v2/${DATASET}/res/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET} 修改；CRF_DIR=/home/mmt/Desktop/deeplab_v2/deeplab-public-ver2/densecrf 修改；if[${DATASET} =="voc2012"]thenIMG_DIR_NAME=VOC_aug/dataset 修改；FEATURE_DIR=/home/mmt/Desktop/deeplab_v2/${DATASET}/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE} 修改；同时把一些不需要的语句都注释掉，要不然容易出错，显示找不到文件。修改后的文件如下：#!/bin/bash############################################ You can either use this script to generate the DenseCRF post-processed results# or use the densecrf_layer (wrapper) in Caffe###########################################DATASET=voc2012LOAD_MAT_FILE=1MODEL_NAME=deeplab_largeFOVTEST_SET=val#val, test# the features folder save the features computed via the model trained with the train set# the features2 folder save the features computed via the model trained with the trainval setFEATURE_NAME=features#features, features2FEATURE_TYPE=fc8# specify the parametersMAX_ITER=10Bi_W=4Bi_X_STD=49Bi_Y_STD=49Bi_R_STD=5Bi_G_STD=5Bi_B_STD=5POS_W=3POS_X_STD=3POS_Y_STD=3######################################## MODIFY THE PATY FOR YOUR SETTING#######################################SAVE_DIR=/home/mmt/Desktop/deeplab_v2/${DATASET}/res/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE}/post_densecrf_W${Bi_W}_XStd${Bi_X_STD}_RStd${Bi_R_STD}_PosW${POS_W}_PosXStd${POS_X_STD}echo"SAVE TO $"CRF_DIR=/home/mmt/Desktop/deeplab_v2/deeplab-public-ver2/densecrf#if [ $ == "voc2012" ]#thenIMG_DIR_NAME=VOC_aug/dataset#elif [ $ == "coco" ]#then# IMG_DIR_NAME=coco#elif [ $ == "voc10_part" ]#then# IMG_DIR_NAME=pascal/VOCdevkit/VOC2012#fi# NOTE THAT the densecrf code only loads ppm imagesIMG_DIR=/home/mmt/Desktop/my_dataset/${IMG_DIR_NAME}/PPMImages#if [ $ == 1 ]#then# the features are saved in .mat formatCRF_BIN=${CRF_DIR}/prog_refine_pascal_v4FEATURE_DIR=/home/mmt/Desktop/deeplab_v2/${DATASET}/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE}#else# the features are saved in .bin format (has called SaveMatAsBin.m in the densecrf/my_script)# CRF_BIN=$/prog_refine_pascal# FEATURE_DIR=/home/mmt/Desktop/deeplab_v2/$/$/$/$/$/bin#fimkdir -p${SAVE_DIR}# run the program${CRF_BIN} -id${IMG_DIR} -fd${FEATURE_DIR} -sd${SAVE_DIR} -i${MAX_ITER} -px${POS_X_STD} -py${POS_Y_STD} -pw${POS_W} -bx${Bi_X_STD} -by${Bi_Y_STD} -br${Bi_R_STD} -bg${Bi_G_STD} -bb${Bi_B_STD} -bw${Bi_W}

进入文件路径，运行程序，结果如下图：

cd ~/Desktop/deeplab_v2/voc2012/

sh sh ./run_densecrf.sh

然后运行

/home/mmt/crf/deeplab-public-ver2/densecrf/my_script/GetDenseCRFResult.m

把bin生成图片格式

注意修改文件路径（GetDenseCRFResult.m，SetupEnv在/deeplab_v2/deeplab-public-ver2/matlab/my_script中）

两个程序的代码如下：

GetDenseCRFResult.m% compute the densecrf result (.bin) to png%addpath('/home/mmt/Desktop/deeplab_v2/deeplab-public-ver2/matlab/my_script');SetupEnv;%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Youdonotneed to change values below%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%ifis_serveriflearn_crf post_folder = sprintf('post_densecrf_W%d_XStd%d_RStd%d_PosW%d_PosXStd%d_ModelType%d_Epoch%d', bi_w, bi_x_std, bi_r_std, pos_w, pos_x_std, model_type, epoch); map_folder = fullfile('/home/mmt/Desktop/deeplab_v2', dataset,'densecrf','res', feature_name, model_name, testset, feature_type, post_folder); save_root_folder = fullfile('/home/mmt/Desktop/deeplab_v2', dataset,'res', feature_name, model_name, testset, feature_type, post_folder); ;elsepost_folder = sprintf('post_densecrf_W%d_XStd%d_RStd%d_PosW%d_PosXStd%d', bi_w, bi_x_std, bi_r_std, pos_w, pos_x_std); map_folder = fullfile('/home/mmt/Desktop/deeplab_v2', dataset,'res', feature_name, model_name, testset, feature_type, post_folder); save_root_folder = map_folder;endelsemap_folder ='../result';endmap_dir = dir(fullfile(map_folder,'*.bin'));fprintf(1,' saving to %s\n', save_root_folder);ifstrcmp(dataset,'voc2012') seg_res_dir = [save_root_folder'/results/VOC2012/'];elseif strcmp(dataset,'coco') seg_res_dir = [save_root_folder,'/results/COCO2014/'];elseerror('Wrong dataset!');endsave_result_folder = fullfile(seg_res_dir,'Segmentation', [id'_'testset'_cls']);if~exist(save_result_folder,'dir') mkdir(save_result_folder);endfori =1:numel(map_dir) fprintf(1,'processing %d (%d)...\n', i, numel(map_dir)); map =LoadBinFile(fullfile(map_folder, map_dir(i).name),'int16'); img_fn = map_dir(i).name(1:end-4); imwrite(uint8(map), colormap, fullfile(save_result_folder, [img_fn,'.png']));end

SetupEnv.m% set up the environment variables%clear all; close all;load('./pascal_seg_colormap.mat');is_server =1;crf_load_mat =1; % the densecrf code loadMATfiles directly (no callSaveMatAsBin.m) % usedONLYbyDownSampleFeature.mlearn_crf =; %NOTUSED.Settois_mat =1; % the results to be evaluated are saved as mat (1)orpng ()has_postprocess =; % has done densecrf post processing (1)ornot()is_argmax =; % the output has been taken argmax already (e.g., coco dataset). % assume the argmax takesC-convention (i.e., start from)debug =; %ifdebug, show some results% vgg128_noup (notoptimized well), akaDeepLab% bi_w =5, bi_x_std =50, bi_r_std =10% vgg128_ms_pool3, akaDeepLab-MSc% bi_w =3, bi_x_std =95, bi_r_std =3% vgg128_noup_pool3_cocomix, akaDeepLab-COCO% bi_w =5, bi_x_std =67, bi_r_std =3%% these are used for the bounding box weak annotation experiments (i.e., to generate the Bbox-Seg)%erode_gt (bbox)% bi_w =41, bi_x_std =33, bi_r_std =4% erode_gt/bboxErode2% bi_w =45, bi_x_std =37, bi_r_std =3, pos_w =15, pos_x_std =3%% initialordefault valuesforcrf%% 这几个参数要修改与run_densecrf.sh中的一致。bi_w = 4; bi_x_std = 49;bi_r_std = 5;pos_w = 3;pos_x_std = 3;%dataset ='voc2012'; %'voc12','coco'修改trainset ='train_aug'; %notusedtestset ='val'; %'val','test'model_name ='deeplab_largeFOV'; % 修改feature_name ='features';feature_type ='fc8'; % fc8 / crfid ='comp6';%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% used for cross-validation%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%rng(10)%downsampling filesforcross-validationdown_sample_method =2; %1:equally sample with"down_sample_rate",2:randomly pick"num_sample"samplesdown_sample_rate =8;num_sample =100; % number of samples usedforcross-validation% rangesforcross-validationrange_pos_w = [3];range_pos_x_std = [3];range_bi_w = [5];range_bi_x_std = [49];range_bi_r_std = [45];

至此，deeplabv2 程序已调试完

感谢ruotianxia的分享！