文章/答案/技术大牛

发布

社区首页 >问答首页 >用OpenMP在macOS上编译Cython

问用OpenMP在macOS上编译Cython
EN

Stack Overflow用户

提问于 2020-10-14 09:13:21

回答 1查看 1.4K关注 0票数 1

我正在学习macOS Mojave 10.14.6，我正在尝试用以下方法从这存储库编译c和c++中所需的扩展模块：

python setup.py build_ext --inplace

这给了我以下错误：

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running build_ext
/Users/user/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py:249: UserWarning: 

                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (g++) is not compatible with the compiler Pytorch was
built with for this platform, which is clang++ on darwin. Please
use clang++ to to compile your extension. Alternatively, you may
compile PyTorch from source using g++, and then you can also use
g++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                              !! WARNING !!

  warnings.warn(WRONG_COMPILER_WARNING.format(

再往下看：

clang: error: unknown argument: '-i'
clang: error: no such file or directory: 'sysroot'
clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
clang: error: unsupported option '-fopenmp'
ninja: build stopped: subcommand failed.

(这应该是相关的部分，最近的跟踪还显示了subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.，最后显示了RuntimeError: Error compiling objects for extension)

从我所了解到的，到目前为止我都试过了：

安装文件中的标志-fopenmp是问题之一，似乎OpenMP丢失了。
因此我安装了brew install llvm和brew install libomp
将PATH=\"/usr/local/bin:$PATH\添加到我的.bash_profile中，希望现在使用新安装的编译器
根据-Xpreprocessor将-fopenmp之前的标志这里添加到安装文件中，从而使extra_compile_args=['-Xpreprocessor', '-std=c99', '-O3', '-fopenmp']
我尝试了CC=gcc python setup.py build_ext --inplace，因为关于将'c‘看作' c++’的警告，但是由于我试图构建c和c++模块，所以我不确定应该将变量设置为什么。
我也尝试了CC=/usr/local/opt/llvm/bin/clang++ python setup.py build_ext --inplace，如建议的这里
尝试在不使用-fopenmp (这实际上不是一个选项，因为我需要由OpenMP IIRC提供的并行执行)的情况下构建它，结果基本上是相同的错误，而没有关于OpenMP的错误。
我还需要设置CXXFLAGS和CFLAGS吗？
现在，我在conda环境中完成了所有这些工作，蟒蛇似乎提供了自己的工具。我应该用这些吗？所需的MacOSX SDK是什么？

我很抱歉这个问题让人困惑，但对编译器没有事先的了解并试图解决这个问题，就像走投无路.我将非常感谢任何关于这种情况的评论，我很高兴提供任何缺失的信息！

更新：

清除.bash_profile并导出export CC='gcc-10'; export CXX='clang++'消除了火炬警告，但clang问题仍然存在
仍然保留在os.environ['CC'] = 'gcc-10'和os.environ['CXX'] = 'clang++中的setup.py

完整的错误消息是：

(torch) [10:48:05] vanessamac: occupancy_networks $ python setup.py build_ext --inplace --verbose
No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
running build_ext
building 'im2mesh.utils.libkdtree.pykdtree.kdtree' extension
Emitting ninja build file /Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/build.ninja...
Compiling objects...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/2] clang++ -MMD -MF '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.o'.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/numpy/core/include -I/Users/vanessamac/miniconda3/envs/torch/include/python3.8 -c -c '/Users/vanessamac/projects/occupancy_networks/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.c' -o '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.o' -std=c99 -O3 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kdtree -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.o 
clang++ -MMD -MF '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.o'.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/numpy/core/include -I/Users/vanessamac/miniconda3/envs/torch/include/python3.8 -c -c '/Users/vanessamac/projects/occupancy_networks/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.c' -o '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/_kdtree_core.o' -std=c99 -O3 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kdtree -D_GLIBCXX_USE_CXX11_ABI=0
clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
clang: error: unsupported option '-fopenmp'
[2/2] clang++ -MMD -MF '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/kdtree.o'.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/numpy/core/include -I/Users/vanessamac/miniconda3/envs/torch/include/python3.8 -c -c '/Users/vanessamac/projects/occupancy_networks/im2mesh/utils/libkdtree/pykdtree/kdtree.c' -o '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/kdtree.o' -std=c99 -O3 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kdtree -D_GLIBCXX_USE_CXX11_ABI=0
FAILED: /Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/kdtree.o 
clang++ -MMD -MF '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/kdtree.o'.d -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/include -arch x86_64 -I/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/numpy/core/include -I/Users/vanessamac/miniconda3/envs/torch/include/python3.8 -c -c '/Users/vanessamac/projects/occupancy_networks/im2mesh/utils/libkdtree/pykdtree/kdtree.c' -o '/Users/vanessamac/projects/occupancy_networks/build/temp.macosx-10.9-x86_64-3.8/im2mesh/utils/libkdtree/pykdtree/kdtree.o' -std=c99 -O3 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=kdtree -D_GLIBCXX_USE_CXX11_ABI=0
clang: warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
clang: error: unsupported option '-fopenmp'
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1509, in _run_ninja_build
    subprocess.run(
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 112, in <module>
    setup(
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/setuptools/__init__.py", line 165, in setup
    return distutils.core.setup(**attrs)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 87, in run
    _build_ext.run(self)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 649, in build_extensions
    build_ext.build_extensions(self)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 208, in build_extension
    _build_ext.build_extension(self, ext)
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/distutils/command/build_ext.py", line 528, in build_extension
    objects = self.compiler.compile(sources,
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 469, in unix_wrap_ninja_compile
    _write_ninja_file_and_compile_objects(
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1228, in _write_ninja_file_and_compile_objects
    _run_ninja_build(
  File "/Users/vanessamac/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1529, in _run_ninja_build
    raise RuntimeError(message)
RuntimeError: Error compiling objects for extension

c++

macos

cython

python

回答 1

Stack Overflow用户

回答已采纳

发布于 2020-10-14 14:20:25

以下是一些提示：

使用gcc，而不是llvm或clang进行无痛的开放-支持在macOS上。请注意，苹果默认的gcc只是苹果clang的别名，您将在gcc --version中看到这一点。你可以用自制软件安装真正的gcc：brew install gcc。
然后在同一个终端窗口内使用export CC='gcc-10' (最新版本应该是gcc 10.x)作为C编译器临时使用自制的gcc。
没有必要设置CXXFLAGS或CFLAGS。所需的标志由setup.py中的distutils/setuptools设置。
您将无法在dmc_cuda_module 10.14.6上编译macOS。最新的macOS版本nvidia为其提供了10.13.6的cuda驱动程序。因此，您可能取消评论这部分的setup.py，并希望最好的，你不需要这个模块.
在使用numpy C时，setup.py中的一些扩展不包括numpy头。在macOS上，有必要为每个扩展包括numpy头，请参阅这句话。因此，您必须将include_dirs=[numpy_include_dir]添加到这些扩展中。
编辑：正如聊天中所讨论的:错误是由于conda忽略CC变量造成的。在通过自制软件安装python+pip和通过pip安装所需的python包之后，这个答案的步骤适用于OP。

总之，这里有一个对我有用的setup.py (macOS 10.5.7，gcc-10)：

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from torch.utils.cpp_extension import BuildExtension, CppExtension, CUDAExtension
import numpy


# Get the numpy include directory.
numpy_include_dir = numpy.get_include()

# Extensions
# pykdtree (kd tree)
pykdtree = Extension(
    'im2mesh.utils.libkdtree.pykdtree.kdtree',
    sources=[
        'im2mesh/utils/libkdtree/pykdtree/kdtree.c',
        'im2mesh/utils/libkdtree/pykdtree/_kdtree_core.c'
    ],
    language='c',
    extra_compile_args=['-std=c99', '-O3', '-fopenmp'],
    extra_link_args=['-lgomp'],
    include_dirs=[numpy_include_dir]
)

# mcubes (marching cubes algorithm)
mcubes_module = Extension(
    'im2mesh.utils.libmcubes.mcubes',
    sources=[
        'im2mesh/utils/libmcubes/mcubes.pyx',
        'im2mesh/utils/libmcubes/pywrapper.cpp',
        'im2mesh/utils/libmcubes/marchingcubes.cpp'
    ],
    language='c++',
    extra_compile_args=['-std=c++11'],
    include_dirs=[numpy_include_dir]
)

# triangle hash (efficient mesh intersection)
triangle_hash_module = Extension(
    'im2mesh.utils.libmesh.triangle_hash',
    sources=[
        'im2mesh/utils/libmesh/triangle_hash.pyx'
    ],
    libraries=['m'],  # Unix-like specific
    include_dirs=[numpy_include_dir]
)

# mise (efficient mesh extraction)
mise_module = Extension(
    'im2mesh.utils.libmise.mise',
    sources=[
        'im2mesh/utils/libmise/mise.pyx'
    ],
)

# simplify (efficient mesh simplification)
simplify_mesh_module = Extension(
    'im2mesh.utils.libsimplify.simplify_mesh',
    sources=[
        'im2mesh/utils/libsimplify/simplify_mesh.pyx'
    ],
    include_dirs=[numpy_include_dir]
)

# voxelization (efficient mesh voxelization)
voxelize_module = Extension(
    'im2mesh.utils.libvoxelize.voxelize',
    sources=[
        'im2mesh/utils/libvoxelize/voxelize.pyx'
    ],
    libraries=['m']  # Unix-like specific
)

# DMC extensions
dmc_pred2mesh_module = CppExtension(
    'im2mesh.dmc.ops.cpp_modules.pred2mesh',
    sources=[
        'im2mesh/dmc/ops/cpp_modules/pred_to_mesh_.cpp',
    ]   
)

# dmc_cuda_module = CUDAExtension(
#     'im2mesh.dmc.ops._cuda_ext', 
#     sources=[
#         'im2mesh/dmc/ops/src/extension.cpp',
#         'im2mesh/dmc/ops/src/curvature_constraint_kernel.cu',
#         'im2mesh/dmc/ops/src/grid_pooling_kernel.cu',
#         'im2mesh/dmc/ops/src/occupancy_to_topology_kernel.cu',
#         'im2mesh/dmc/ops/src/occupancy_connectivity_kernel.cu',
#         'im2mesh/dmc/ops/src/point_triangle_distance_kernel.cu',
#     ]
# )

# Gather all extension modules
ext_modules = [
    pykdtree,
    mcubes_module,
    triangle_hash_module,
    mise_module,
    simplify_mesh_module,
    voxelize_module,
    dmc_pred2mesh_module,
    #dmc_cuda_module,
]

setup(
    ext_modules=cythonize(ext_modules),
    cmdclass={
        'build_ext': BuildExtension
    }
)

票数 2

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/64350106

复制

相似问题

问用OpenMP在macOS上编译Cython
EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问用OpenMP在macOS上编译CythonEN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问用OpenMP在macOS上编译Cython
EN