OpenBLAS:改进OpenBLASConfig.cmake生成方式,解除cmake脚本的路径依赖

问题描述

当我们用OpenBLAS提供的Makefile进行编译,并用make install PREFIX=/you/install/path,安装到指定的路径时,你会发现,/you/install/path/lib/cmake/openblas/OpenBLASConfig.cmake文件中OpenBLAS_INCLUDE_DIRS和OpenBLAS_LIBRARIES的值都是绝对路径,如下:

/you/install/path/lib/cmake/openblas/OpenBLASConfig.cmake (linux下生成动态库)

SET(OpenBLAS_VERSION "0.2.18")
SET(OpenBLAS_INCLUDE_DIRS /home/hadoop/Desktop/caffe-static/release/OpenBLAS_linux_x86_64/include)
SET(OpenBLAS_LIBRARIES /home/hadoop/Desktop/caffe-static/release/OpenBLAS_linux_x86_64/bin/libopenblas.so)

(linux下生成静态库)

SET(OpenBLAS_VERSION "0.2.18")
SET(OpenBLAS_INCLUDE_DIRS /home/hadoop/Desktop/caffe-static/release/OpenBLAS_linux_x86_64/include)
SET(OpenBLAS_LIBRARIES /home/hadoop/Desktop/caffe-static/release/OpenBLAS_linux_x86_64/lib/libopenblas.a)

(windows下生成动态库)

SET(OpenBLAS_VERSION "0.2.18")
SET(OpenBLAS_INCLUDE_DIRS D:/caffe-static/release/OpenBLAS_windows_vc140_x86_mt/include)
SET(OpenBLAS_LIBRARIES D:/caffe-static/release/OpenBLAS_windows_vc140_x86_mt/bin/libopenblas.dll)

关于windows下如何用MinGW编译OpenBLAS,参见我的另一篇博客《Windows下MSYS2中编译OpenBLAS过程记录》

如果你编译OpenBLAS只是在本机使用,那没啥问题,如果想把这个编译好的OpenBLAS 移到别的位置或提供给别人使用,并且也是用cmake编译项目。那么问题就来了,因为OpenBLASConfig.cmake中的路径依赖问题,导致cmake脚本中调用find_package( OpenBLAS NO_MODULE )返回的OpenBLAS_LIBRARIES和OpenBLAS_LIBRARIES的值都是错的!

而且你发现没有?上面Windows下生成的OpenBLASConfig.cmake中OpenBLAS_LIBRARIES的值居然是.dll的路径。而不是动态库导入库(import library)(.dll.a)的路径,也就是说OpenBLAS在生成OpenBLASConfig.cmake没有考虑到linux和windows的区别,只是按linux的方式将OpenBLAS_LIBRARIES置为静态库/动态库(.a,.so,dll)的路径。

如果你是用手工在Visual Studio中创建工程并设置工程参数,那么这都不是问题,对你没有任何影响,但如果你像我一样,用cmake进行工程编译,这是个很要命的问题,因为cmake根据这些错误的信息无法生成正确的Makefile,当然也不能正常编译。

原因分析

怎么办呢?指望OpenBLAS官方修改这个问题怕是来不及了,只能自己想办法解决。 于是我仔细研究了OpenBLAS的Makefile的体系。算是基本搞清楚了OpenBLAS源码下那一大堆Makefile.*的作用。最终找到了问题的关键点: Makefile.install 这个文件是OpenBLAS的安装脚本,当执行 make install时其实就是执行这个脚本来执行安装工作的。

而在这其中生成OpenBLASConfig.cmake的代码就是如下几行(line 97-116 OpenBLAS 0.2.18) (中文注释为本文作者所加)

#Generating OpenBLASConfig.cmake
    @echo Generating $(OPENBLAS_CMAKE_CONFIG) in $(DESTDIR)$(OPENBLAS_CMAKE_DIR)
    # 对应OpenBLASConfig.cmake中line 1
    @echo "SET(OpenBLAS_VERSION \"${VERSION}\")" > $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)    
    # 对应OpenBLASConfig.cmake中line 2
    @echo "SET(OpenBLAS_INCLUDE_DIRS ${OPENBLAS_INCLUDE_DIR})" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)

ifndef NO_SHARED
#ifeq logical or
# 判断操作系统将OpenBLAS_LIBRARIES置为动态库文件路径
ifeq ($(OSNAME), $(filter $(OSNAME),Linux FreeBSD NetBSD))
    # 对应OpenBLASConfig.cmake中line 3
    @echo "SET(OpenBLAS_LIBRARIES ${OPENBLAS_LIBRARY_DIR}/$(LIBPREFIX).so)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
ifeq ($(OSNAME), $(filter $(OSNAME),WINNT CYGWIN_NT))
    # 对应OpenBLASConfig.cmake中line 3
    # 这里 LIBDLLNAME   = $(LIBPREFIX).dll 参见 Makefile.system
    @echo "SET(OpenBLAS_LIBRARIES ${OPENBLAS_BINARY_DIR}/$(LIBDLLNAME))" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
ifeq ($(OSNAME), Darwin)
    # 对应OpenBLASConfig.cmake中line 3
    @echo "SET(OpenBLAS_LIBRARIES ${OPENBLAS_LIBRARY_DIR}/$(LIBPREFIX).dylib)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
else
# 将OpenBLAS_LIBRARIES置为静态库文件路径
#only static
    # 对应OpenBLASConfig.cmake中line 3
    @echo "SET(OpenBLAS_LIBRARIES ${OPENBLAS_LIBRARY_DIR}/$(LIBPREFIX).$(LIBSUFFIX))" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif

OPENBLAS_INCLUDE_DIR,OPENBLAS_LIBRARY_DIR,OPENBLAS_BINARY_DIR在Makefile.install文件开头定义 (line 8-10 OpenBLAS 0.2.18)

上面的三行@echo的代码清晰的对应着OpenBLASConfig.cmake中的三行代码,由此我们理解为什么OpenBLASConfig.cmake中的路径都是绝对路径,而且windows下OpenBLAS_LIBRARIES的值是.dll文件的路径了。

解决方法

原因搞清楚就好办了,要解决问题,只要修改上面这段代码,改进生成OpenBLASConfig.cmake的逻辑就可以了。

以下是上面代码(Makefile.install)修改后的版本:

#Generating OpenBLASConfig.cmake
# modified by guyadong for windows build
    @echo Generating $(OPENBLAS_CMAKE_CONFIG) in $(DESTDIR)$(OPENBLAS_CMAKE_DIR)  
    @echo "SET(OpenBLAS_VERSION \"${VERSION}\")" > $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)  
    @echo "# modified by guyadong,use relative path instead of absolute path" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)   
    @echo "# Compute the installation prefix relative to this file." >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)    
    @echo "get_filename_component(_IMPORT_PREFIX \"\$${CMAKE_CURRENT_LIST_FILE}\" PATH)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "get_filename_component(_IMPORT_PREFIX \"\$${_IMPORT_PREFIX}\" PATH)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)  
    @echo "get_filename_component(_IMPORT_PREFIX \"\$${_IMPORT_PREFIX}\" PATH)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)  
    @echo "get_filename_component(_IMPORT_PREFIX \"\$${_IMPORT_PREFIX}\" PATH)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)  
    @echo "if(_IMPORT_PREFIX STREQUAL \"/\")" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " set(_IMPORT_PREFIX \"\")" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "endif()" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "SET(OpenBLAS_INCLUDE_DIRS \$${_IMPORT_PREFIX}/include)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)

ifndef NO_SHARED
#ifeq logical or
ifeq ($(OSNAME), $(filter $(OSNAME),Linux FreeBSD NetBSD))
    @echo "SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/lib/$(LIBPREFIX).so)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
ifeq ($(OSNAME), $(filter $(OSNAME),WINNT CYGWIN_NT))
# windows下编译时生成条件判断语句
    @echo "# use dynamic import library if MSVC" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)    
    @echo "if(MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/lib/$(LIBDLLNAME).a)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "else(MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/bin/$(LIBDLLNAME))" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "endif(MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "SET(OPENBLAS_IS_DYNAMIC true)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
ifeq ($(OSNAME), Darwin)
    @echo "SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/lib/$(LIBPREFIX).dylib)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
else
#only static
    @echo "SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/lib/$(LIBPREFIX).$(LIBSUFFIX))" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
# windows下编译时生成条件判断语句
ifeq ($(OSNAME), $(filter $(OSNAME),WINNT CYGWIN_NT))    
    @echo "if(MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " message(STATUS \"OpenBLAS_LIBRARIES(static library):\$${OpenBLAS_LIBRARIES}\" )" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " message(FATAL_ERROR \"With Visual Studio, only dynamic linking is supported,please reinstall OpenBLAS without NO_STATIC=1\" )" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "endif(MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "SET(OPENBLAS_IS_DYNAMIC false)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif   
endif
ifeq ($(OSNAME), $(filter $(OSNAME),WINNT CYGWIN_NT))    
    @echo "# if OpenBlas_USE_STATIC is true,force to use static library with MinGW" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "if(OPENBLAS_IS_DYNAMIC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " if(EXISTS \"\$${_IMPORT_PREFIX}/lib/$(LIBPREFIX).$(LIBSUFFIX)\" AND OpenBlas_USE_STATIC AND NOT MSVC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "     MESSAGE(STATUS \"use OpenBlas static library\")" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "     SET(OpenBLAS_LIBRARIES \$${_IMPORT_PREFIX}/lib/$(LIBPREFIX).$(LIBSUFFIX))" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo " endif()" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "endif(OPENBLAS_IS_DYNAMIC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
    @echo "unset(OPENBLAS_IS_DYNAMIC)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)
endif
    @echo "unset(_IMPORT_PREFIX)" >> $(DESTDIR)$(OPENBLAS_CMAKE_DIR)/$(OPENBLAS_CMAKE_CONFIG)

经过上面的修改,再重新执行make install PREFIX=/you/install/path,生成的OpenBLASConfig.cmake(中文注释为本文作者所加) (linux下生成静态态库)

# modified by guyadong,use relative path instead of absolute path
# Compute the installation prefix relative to this file.
# 三次调用 get_filename_component 计算 OpenBLAS安装路径
# _IMPORT_PREFIX为OpenBLAS安装路径
get_filename_component(_IMPORT_PREFIX "${CMAKE_CURRENT_LIST_DIR}" PATH)
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
if(_IMPORT_PREFIX STREQUAL "/")
    set(_IMPORT_PREFIX "")
endif()
SET(OpenBLAS_INCLUDE_DIRS ${_IMPORT_PREFIX}/include)
SET(OpenBLAS_LIBRARIES ${_IMPORT_PREFIX}/lib/libopenblas.a)
unset(_IMPORT_PREFIX)

(windows下生成动态库)

# modified by guyadong,use relative path instead of absolute path
# Compute the installation prefix relative to this file.
# 三次调用 get_filename_component 计算 OpenBLAS安装路径
# _IMPORT_PREFIX为OpenBLAS安装路径
get_filename_component(_IMPORT_PREFIX "${CMAKE_CURRENT_LIST_DIR}" PATH)
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
get_filename_component(_IMPORT_PREFIX "${_IMPORT_PREFIX}" PATH)
if(_IMPORT_PREFIX STREQUAL "/")
    set(_IMPORT_PREFIX "")
endif()
SET(OpenBLAS_INCLUDE_DIRS ${_IMPORT_PREFIX}/include)
# 判断编译器类型,MSVC下置为import library(.dll.a) ,否则(MinGW)置为.dll
if(MSVC)
    SET(OpenBLAS_LIBRARIES ${_IMPORT_PREFIX}/lib/libopenblas.dll.a)
else(MSVC)
    SET(OpenBLAS_LIBRARIES ${_IMPORT_PREFIX}/bin/libopenblas.dll)
endif(MSVC)
SET(openblas_is_dynamic true)
# if OpenBlas_USE_STATIC is true,force to use static library with MinGW
if(openblas_is_dynamic)
    # 使用MinGW编译时,如果定义了OpenBlas_USE_STATIC,则置为静态库
    if(EXISTS "${_IMPORT_PREFIX}/lib/libopenblas.a" AND OpenBlas_USE_STATIC AND NOT MSVC)
        SET(OpenBLAS_LIBRARIES ${_IMPORT_PREFIX}/lib/libopenblas.a)
    endif()
endif(openblas_is_dynamic)
unset(openblas_is_dynamic)
unset(_IMPORT_PREFIX)

完整代码

修改后的OpenBLAS Makefile.install完整代码参见 :

https://gitee.com/l0km/caffe-static/tree/master/patch/OpenBLAS-0.2.18/Makefile.install

后记

本文对 Makefile.install的修改已经申请 Pull Request ,如果被接受,后续OpenBLAS的版本就不需要这样手工修改了。 https://github.com/xianyi/OpenBLAS/pull/1231

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏精讲JAVA

接口方法上的注解无法被 @Aspect 声明的切面拦截的原因分析

在Spring中使用MyBatis的Mapper接口自动生成时,用一个自定义的注解标记在Mapper接口的方法中,再利用@Aspect定义一个切面,拦截这个注解...

1462
来自专栏码匠的流水账

聊聊spring cloud gateway的GatewayFilter

本文主要研究一下spring cloud gateway的GatewayFilter

1931
来自专栏码匠的流水账

聊聊eureka client的fetch-remote-regions-registry属性

本文主要研究一下eureka client的fetch-remote-regions-registry属性

571
来自专栏JAVA后端开发

给mybatis添加自动建表,自动加字段的功能

以前项目用惯了hibernate,jpa,它有个自动建表功能,只要在PO里加上配置就可以了,感觉很爽. 但现在用mybatis,发现没有该功能,每次都加个字段...

1753
来自专栏美团技术团队

【美团技术团队博客】Dive into Category

本文系学习Objective-C的runtime源码时候整理所成,主要剖析了category在runtime层的实现原理以及和category相关的方方面面,包...

3456
来自专栏Java编程技术

常用开源框架中设计模式使用分析- 装饰器模式(Decorator Pattern)

装饰器模式是一种结构性模式,它作用是对对象已有功能进行增强,但是不改变原有对象结构。这避免了通过继承方式进行功能扩充导致的类体系臃肿。

711
来自专栏张善友的专栏

线程安全的Generic Dictionary

System.Collections.Generic.Dictionary<,> 只要不修改该集合,Dictionary 就可以同时支持多个阅读器。即便如此,从...

1996
来自专栏salesforce零基础学习

salesforce 零基础学习(十七)Trigger用法

看本篇之前可以相应阅读以下Trigger相关文章: 1.https://developer.salesforce.com/page/Trigger_Framew...

3256
来自专栏高性能服务器开发

后台C++开发你一定要知道的条件变量

今天因为工作需要,需要帮同事用C语言(不是C++)写一个生产者消费者的任务队列工具库,考虑到不能使用任何第三库和C++的任何特性,所以我将任务队列做成一个链表,...

3084
来自专栏mini188

学习AOP之深入一点Spring Aop

上一篇《学习AOP之认识一下SpringAOP》中大体的了解了代理、动态代理及SpringAop的知识。因为写的篇幅长了点所以还是再写一篇吧。接下来开始深入一点...

1859

扫码关注云+社区