linux下用cmake对caffe静态编译时-static-libstdc++参数无效的问题

以下是用于cmake 生成 Makefile文件对Caffe进行静态库连接编译的shell脚本,

#!/bin/bash
# cmake 静态编译 caffe-ssd 代码脚本
# author guyadong@gdface.net
shell_folder=$(cd "$(dirname "$0")";pwd)
. $shell_folder/build_funs
. $shell_folder/build_vars
# 项目安装路径
install_path=$SSD_INSTALL_PATH
echo install_path:$install_path
# gflags 安装路径
gflags_root=$GFLAGS_INSTALL_PATH
exit_if_not_exist "$gflags_root/lib/libgflags.a" "not found $gflags_root/lib/libgflags.a,please build $GFLAGS_PREFIX"
# glog 安装路径
glog_root=$GLOG_INSTALL_PATH
exit_if_not_exist "$glog_root/lib/libglog.a" "not found $glog_root/lib/libglog.a,please build $GLOG_PREFIX"
# hdf5 cmake 位置  
hdf5_cmake_dir=$HDF5_INSTALL_PATH/share/cmake
exit_if_not_exist $hdf5_cmake_dir "not found $hdf5_cmake_dir,please build $HDF5_PREFIX"
exit_if_not_exist $BOOST_INSTALL_PATH "not found $BOOST_INSTALL_PATH,please build $BOOST_PREFIX"
exit_if_not_exist $OPENBLAS_INSTALL_PATH "not found $OPENBLAS_INSTALL_PATH,please build $OPENBLAS_PREFIX"
exit_if_not_exist $PROTOBUF_INSTALL_PATH "not found $PROTOBUF_INSTALL_PATH,please build $PROTOBUF_PREFIX"
# protobuf lib 路径,
# centos下安装路径可能是lib64
[ -e "$PROTOBUF_INSTALL_PATH/lib" ] && protobuf_lib=$PROTOBUF_INSTALL_PATH/lib
[ -e "$PROTOBUF_INSTALL_PATH/lib64" ] && protobuf_lib=$PROTOBUF_INSTALL_PATH/lib64
exit_if_not_exist $SNAPPY_INSTALL_PATH "not found $SNAPPY_INSTALL_PATH,please build $SNAPPY_PREFIX"
# lmdb 安装路径根目录
lmdb_install_root=$LMDB_INSTALL_PATH/usr/local
exit_if_not_exist $lmdb_install_root "not found $lmdb_install_root,please build lmdb"
exit_if_not_exist $LEVELDB_INSTALL_PATH "not found $LEVELDB_INSTALL_PATH,please build $LEVELDB_PREFIX"
# opencv 配置文件(OpenCVConfig.cmake)所在路径
opencv_cmake_dir=$OPENCV_INSTALL_PATH/share/OpenCV
exit_if_not_exist $opencv_cmake_dir "not found $opencv_cmake_dir,please build $OPENCV_PREFIX"

pushd $SOURCE_ROOT/$SSD_FOLDER
clean_folder build.gcc
#mkdir_if_not_exist build.gcc
pushd build.gcc
# 指定 OpenBLAS 安装路径 参见 $caffe_source/cmake/Modules/FindOpenBLAS.cmake
export OpenBLAS_HOME=$OPENBLAS_INSTALL_PATH
# 指定 lmdb 安装路径 参见 $caffe_source/cmake/Modules/FindLMDB.cmake.cmake
export LMDB_DIR=$lmdb_install_root
# 指定 leveldb 安装路径 参见 $caffe_source/cmake/Modules/FindLevelDB.cmake.cmake
export LEVELDB_ROOT=$LEVELDB_INSTALL_PATH
# GLOG_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindGlog.cmake
# GFLAGS_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindGFlags.cmake
# HDF5_ROOT 参见 https://cmake.org/cmake/help/v3.8/module/FindHDF5.html
# BOOST_ROOT,Boost_NO_SYSTEM_PATHS 参见 https://cmake.org/cmake/help/v3.8/module/FindBoost.html
# SNAPPY_ROOT_DIR 参见 $caffe_source/cmake/Modules/FindSnappy.cmake
# PROTOBUF_LIBRARY,PROTOBUF_PROTOC_LIBRARY... 参见 https://cmake.org/cmake/help/v3.8/module/FindProtobuf.html
# OpenCV_DIR 参见https://cmake.org/cmake/help/v3.8/command/find_package.html
$CMAKE_EXE "$(dirs +1)" $CMAKE_VARS_DEFINE -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$install_path \
    -DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc" \
    -DGLOG_ROOT_DIR=$glog_root \
    -DGFLAGS_ROOT_DIR=$gflags_root \
    -DHDF5_ROOT=$HDF5_INSTALL_PATH \
    -DBOOST_ROOT=$BOOST_INSTALL_PATH \
    -DBoost_NO_SYSTEM_PATHS=on \
    -DSNAPPY_ROOT_DIR=$SNAPPY_INSTALL_PATH \
    -DOpenCV_DIR=$opencv_cmake_dir \
    -DPROTOBUF_LIBRARY=$protobuf_lib/libprotobuf.a \
    -DPROTOBUF_PROTOC_LIBRARY=$protobuf_lib/libprotoc.a \
    -DPROTOBUF_LITE_LIBRARY=$protobuf_lib/libprotobuf-lite.a \
    -DPROTOBUF_PROTOC_EXECUTABLE=$PROTOBUF_INSTALL_PATH/bin/protoc \
    -DPROTOBUF_INCLUDE_DIR=$PROTOBUF_INSTALL_PATH/include \
    -DCPU_ONLY=ON \
    -DBLAS=Open \
    -DBUILD_SHARED_LIBS=off \
    -DBUILD_docs=off \
    -DBUILD_python=off \
    -DBUILD_python_layer=off \
    -DUSE_LEVELDB=on \
    -DUSE_LMDB=on \
    -DUSE_OPENCV=on 
exit_on_error
remove_if_exist $install_path
make -j $MAKE_JOBS install
exit_on_error
popd
rm -fr build.gcc
popd

在脚本中,调用cmake生成Makefile时,添加了-DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static-libgcc"参数用于指定 将libstdc++,libgcc以静态库形式连接,然而在实际使用过程发现当指定 USE_OPENCV=on(即使用OpenCV,opencv已经预先做了静态库编译)时,编译出的caffe,用ldd查看还是会依赖libstdc++.so和libgcc.so,也就是说-static-libstdc++ -static-libgcc无效了。。。 USE_OPENCV=off时则能正常静态连接。

这个问题困扰了几天,后来通过比较<project>.dir下的link.txt(cmake生成的),发现,当USE_OPENCV=on时生成的link.txt中,自动在opencv静态库加了-lstdc++ 参数。。。 如下是caffe.bin项目link.txt(./tools/CMakeFiles/caffe.bin.dir/link.txt),太长我加了分行符:

/usr/local/bin/g++   -fPIC -Wall -Wno-sign-compare -Wno-uninitialized -O3 -DNDEBUG  -static-libstdc++ -static-libgcc -rdynamic CMakeFiles/caffe.bin.dir/caffe.cpp.o  -o caffe -Wl,-rpath,:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -Wl,--whole-archive ../lib/libcaffe.a -Wl,--no-whole-archive ../lib/libproto.a 
/home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_system.a 
/home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_thread.a 
/home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_filesystem.a 
/home/hadoop/tmp/dl/depends/release/boost_linux_x86_64/lib/libboost_regex.a -lpthread 
/home/hadoop/tmp/dl/depends/release/glog_linux_x86_64/lib/libglog.a 
/home/hadoop/tmp/dl/depends/release/gflags_linux_x86_64/lib/libgflags.a 
/home/hadoop/tmp/dl/depends/release/protobuf_linux_x86_64/lib64/libprotobuf.a -lpthread 
/home/hadoop/tmp/dl/depends/release/glog_linux_x86_64/lib/libglog.a 
/home/hadoop/tmp/dl/depends/release/gflags_linux_x86_64/lib/libgflags.a 
/home/hadoop/tmp/dl/depends/release/protobuf_linux_x86_64/lib64/libprotobuf.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_cpp.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl_cpp.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl.a /home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_cpp.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl_cpp.a 
/home/hadoop/tmp/dl/depends/release/hdf5_linux_x86_64/lib/libhdf5_hl.a 
/home/hadoop/tmp/dl/depends/release/lmdb_linux_x86_64/usr/local/lib/liblmdb.a 
/home/hadoop/tmp/dl/depends/release/leveldb_linux_x86_64/lib/libleveldb.a 
/home/hadoop/tmp/dl/depends/release/snappy_linux_x86_64/lib/libsnappy.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_highgui.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibjpeg.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibpng.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibtiff.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/liblibjasper.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/libIlmImf.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_imgproc.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/lib/libopencv_core.a 
/home/hadoop/tmp/dl/depends/release/opencv_linux_x86_64/share/OpenCV/3rdparty/lib/libzlib.a 
-lstdc++ -lm -lpthread -lrt 
/home/hadoop/tmp/dl/depends/release/OpenBLAS_linux_x86_64/lib/libopenblas.a 
-ldl -pthread -lrt 

就是这尾部的-lstdc++参数导致-static-libstdc++ -static-libgcc无效,尝试手工删除 -lstdc++,则编译通过,stdc++,libgcc都能静态连接进来了。于是果然在cmake生成Makefile后,添加了如下代码,则问题解决:

# 修改所有 link.txt 删除-lstdc++ 选项,保证静态连接libstdc++库,否则在USE_OPENCV=on的情况下,libstdc++不会静态链接
for file in $(find . -name link.txt)
do 
    echo "modifing file: $file"
    #sed -i -r "s/-lstdc\+\+/ /g" $file
done

静态编译后,ldd 查看bin下的caffe依赖关系显示如下,除了系统库之外,不再依赖任何其他动态库,文件大小也达到28MB。

[hadoop@t2-centos6 bin]$ ldd ../release/caffe-ssd_linux_x86_64/bin/caffe linux-vdso.so.1 => (0x00007fff8fb58000) libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c24600000) librt.so.1 => /lib64/librt.so.1 (0x0000003c24e00000) libdl.so.2 => /lib64/libdl.so.2 (0x0000003c24a00000) libm.so.6 => /lib64/libm.so.6 (0x0000003c25200000) libc.so.6 => /lib64/libc.so.6 (0x0000003c24200000) /lib64/ld-linux-x86-64.so.2 (0x0000003c23e00000)

但是为什么opencv的库会导致这个问题。一直没搞明白。

本文贴出的脚本并不完整 关于caffe静态编译的完整脚本,请从从csdn CODE获取: https://code.csdn.net/10km/caffe-static

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏小狼的世界

SMTP的相关命令

邮件的发送,主要是通过SMTP协议来实现的。SMTP协议最早在RFC 821(1982年)中定义,最后更新是在RFC 5321(2008年)中,更新中包含了扩展...

892
来自专栏运维

vSphere初体验之新建集群

物理机:DELL  IPTIPLEX 990 PC机 I52400处理器支持VT功能  8G内存 千兆网卡

701
来自专栏gaoqin31

centos6.5编译安装LNMP架构web环境

作为 Web 服务器:相比 Apache,Nginx 使用更少的资源,支持更多的并发连接,体现更高的效率。

1513
来自专栏一只程序汪的自我修养

每周开源项目分享-年轻人的第一个OAuth2.0 Server:hydra

文档地址:https://www.ory.sh/docs/guides/master/hydra/

1452
来自专栏人工智能LeadAI

配置深度学习主机与环境(TensorFlow+1080Ti) | 第二章 Win10&Ubuntu双系统与显卡驱动安装

网上安装双系统的教程不少,但多数教程所使用的硬件以现在的眼光看来显得有些过时;另外,其原有所使用的方法,对于新的硬件也不再合适。本教程写于2017年7月,希望能...

4686
来自专栏闵开慧

WINDOWS下使用虚拟机安装ubuntu及其上网设置的安装说明

LINUX是个好东西,可是如果是新手.而且是想多系统共存的话,安装就会有点难。建议初装LINUX的朋友试试用虚拟机VMware workstation安装...

40811
来自专栏糊一笑

小白学Docker之Swarm

承接上篇文章:小白学Docker之基础篇,自学网站来源于https://docs.docker.com/get-started 系列文章: 小白学Docker之...

40411
来自专栏大魏分享(微信公众号:david-share)

OpenShift企业版安装:单Master集群

OpenShift企业版安装:单Master集群 项目描述本文目的本文描述搭建红帽OpenShift容器平台单Master集群的过程。 适合用于在没有互联网连接...

1.4K3
来自专栏北京马哥教育

linux安全小贴士

对于系统管理员,让产品的系统更安全,免于骇客和黑客的攻击,一直是一项挑战。这是我们关于“如何让Linux系统更安全” 或者 “加固Linux系统“之类话题的第一...

2668
来自专栏bboysoul

linux编译安装mysql5.7

wget mirrors.sohu.com/mysql/MySQL-5.7/mysql-boost-5.7.17.tar.gz 因为官方下载地址速度太慢了,所...

793

扫码关注云+社区