Opentelemetry——分析C++项目链接时循环依赖导致的错误

方亮

发布于 2024-05-24 19:22:32

890

发布于 2024-05-24 19:22:32

文章被收录于专栏：方亮

大纲

环境
分析过程
- 函数是否真的未定义
- - 是否有完整实现
  - 被谁编译
  - 代码是否被编译到静态库
- 链接出现了什么问题
- - 原因猜想
解决方案
参考资料

在《Opentelemetry-Language APIs & SDKs-C+±Getting Started》一文中，介绍了如果编译一个可以发出Trace遥测数据的C++项目。虽然过程很详细，但是在我的环境下，编译出现了问题。本文将介绍分析并解决该问题的过程。

环境

我的环境是

cat /proc/version

Linux version 5.15.0-102-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024

cmake --version

cmake version 3.29.2 CMake suite maintained and supported by Kitware (kitware.com/cmake).

Opentelemetry-cpp的编译需要3.20以上的cmake。如果操作系统比较新，直接apt安装最新的cmake基本能满足需求；如果比较老，软件安装包里也没有符合的cmake。则可以参考这篇文章《正确的方式升级ubuntu的cmake》。

g++ --version

g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright © 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

分析过程

在执行完《Opentelemetry-Language APIs & SDKs-C+±Getting Started》中最后一条编译指令后，会报出如下错误：

/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogLevel()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateSpanId()':
random_id_generator.cc:(.text+0x41): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateTraceId()':
random_id_generator.cc:(.text+0xc7): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::OStreamSpanExporter(std::ostream&)':
span_exporter.cc:(.text+0x19a): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::~OStreamSpanExporter()':
span_exporter.cc:(.text._ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD2Ev[_ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD5Ev]+0x36): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::~SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o):(.data.rel.ro._ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE[_ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE]+0x10): undefined reference to `typeinfo for opentelemetry::v1::sdk::trace::SpanExporter'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a(resource_detector.cc.o): in function `opentelemetry::v1::sdk::resource::OTELResourceDetector::Detect()':
resource_detector.cc:(.text+0x5f): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
/usr/bin/ld: resource_detector.cc:(.text+0x7e): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/dice-server.dir/build.make:102: dice-server] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/dice-server.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

问题比较多，我们先定位和关注第一个问题：

函数是否真的未定义

/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'

它的意思是找不到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()的定义。

我们到工程中定位该文件，然后看它编译到哪个项目里去了。

是否有完整实现

这个函数定义在opentelemetry-cpp/sdk/src/common/global_log_handler.cc中。可以看到该函数是有完整实现的。

被谁编译

我们在opentelemetry-cpp/sdk/src/common/global_log_handler.cc所在目录下找到CMakeLists.txt文件

其中编译相关的指令是

……
set(COMMON_SRCS random.cc core.cc global_log_handler.cc env_variables.cc
                base64.cc)
……
add_library(opentelemetry_common ${COMMON_SRCS})

set_target_properties(opentelemetry_common PROPERTIES EXPORT_NAME common)
set_target_version(opentelemetry_common)

target_link_libraries(
  opentelemetry_common PUBLIC opentelemetry_api opentelemetry_sdk
                              Threads::Threads)

可见这个函数被编译到opentelemetry_common这个静态库中。

这个文件位于opentelemetry-cpp/build/sdk/src/common/libopentelemetry_common.a。

代码是否被编译到静态库

nm libopentelemetry_common.a | grep GetHandlerAndLevel

0000000000000048 b _ZGVZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEvE17handler_and_level
000000000000025a T _ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv
0000000000000020 b _ZZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEvE17handler_and_level
                 U _ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv

第二行的符号_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv指向的就是opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法（因为这是C++项目，所以符号表是经过处理的），它的状态是T，即

The symbol is in the text (code) section.

这说明这个方法的实现是存在于libopentelemetry_common.a中的。

所以，这只能说明roll-dice的链接过程，没有找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法，而方法的确在静态库libopentelemetry_common.a中。进而有两个可能：

roll-dice没有链接libopentelemetry_common.a
roll-dice由于种种原因没有在libopentelemetry_common.a中找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()的实现。

链接出现了什么问题

我们先研究上述1的可能性，即roll-dice是否没有链接libopentelemetry_common.a？

在roll-dice/build/CMakeFiles/dice-server.dir/link.txt文件中，我们看到如下内容

/usr/bin/c++ -rdynamic "CMakeFiles/dice-server.dir/main.cpp.o" 
	-o dice-server  /home/fangliang/otel-cpp-starter/oatpp/build/src/liboatpp.a 
	/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/common/libopentelemetry_common.a 
	/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a 
	/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a 
	/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a

可以发现，roll-dice项目链接了libopentelemetry_common.a 。

那只能去研究为什么roll-dice没在libopentelemetry_common.a中找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法。

我们回到最开的错误提示，需要梳理下它们的关系

/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogLevel()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateSpanId()':
random_id_generator.cc:(.text+0x41): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateTraceId()':
random_id_generator.cc:(.text+0xc7): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'

这些都提示libopentelemetry_trace.a依赖于libopentelemetry_common.a，但是没找到libopentelemetry_common.a中的的确存在的一系列方法。

/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::OStreamSpanExporter(std::ostream&)':
span_exporter.cc:(.text+0x19a): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::~OStreamSpanExporter()':
span_exporter.cc:(.text._ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD2Ev[_ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD5Ev]+0x36): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::~SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o):(.data.rel.ro._ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE[_ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE]+0x10): undefined reference to `typeinfo for opentelemetry::v1::sdk::trace::SpanExporter'

这些都提示libopentelemetry_exporter_ostream_span.a依赖于libopentelemetry_trace.a，但是没找到libopentelemetry_trace.a中的的确存在的一系列方法。

/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a(resource_detector.cc.o): in function `opentelemetry::v1::sdk::resource::OTELResourceDetector::Detect()':
resource_detector.cc:(.text+0x5f): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
/usr/bin/ld: resource_detector.cc:(.text+0x7e): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'

这些都提示libopentelemetry_resources.a依赖于libopentelemetry_common.a，但是没找到libopentelemetry_common.a中的的确存在的一系列方法。

这些我们在CMakelists.txt中也存在也会得到印证：

opentelemetry-cpp/sdk/src/trace/CMakeLists.txt

target_link_libraries(opentelemetry_trace PUBLIC opentelemetry_common
                                                 opentelemetry_resources)

opentelemetry-cpp/exporters/ostream/CMakeLists.txt

target_link_libraries(opentelemetry_exporter_ostream_span
                      PUBLIC opentelemetry_trace)

opentelemetry-cpp/sdk/src/resource/CMakeLists.txt

target_link_libraries(opentelemetry_resources opentelemetry_common)

而roll-dice的链接指令顺序是

原因猜想

这个顺序似乎符合一种猜想：

链接opentelemetry_common时不知道opentelemetry_trace需要什么，导致后续链接opentelemetry_trace时找不到依赖opentelemetry_common中的方法。
链接opentelemetry_trace时不知道opentelemetry_exporter_ostream_span需要什么，导致后续链接opentelemetry_exporter_ostream_span时找不到依赖opentelemetry_trace中的方法。
链接opentelemetry_common时不知道opentelemetry_resources需要什么，导致后续链接opentelemetry_resources时找不到依赖opentelemetry_common中的方法。

这个猜想和之前发现的报错信息吻合。

解决方案

那么我们将链接顺序做个调整：

opentelemetry_common被依赖最多，最后链接
opentelemetry_resources只依赖于opentelemetry_common，但是被opentelemetry_trace依赖，所以要位于opentelemetry_trace之后
opentelemetry_trace被opentelemetry_exporter_ostream_span依赖，所以它要在opentelemetry_exporter_ostream_span之后链接，而在依赖项opentelemetry_resources和opentelemetry_common之前。

我们只需要修改roll-dice/CMakeLists.txt文件即可

# target_link_libraries(dice-server PRIVATE ${OATPP_LIB} ${OPENTELEMETRY_COMMON_LIB} ${OPENTELEMETRY_TRACE_LIB} ${OPENTELEMETRY_EXPORTER_LIB} ${OPENTELEMETRY_RESOURCE_LIB})
target_link_libraries(dice-server PRIVATE ${OATPP_LIB} ${OPENTELEMETRY_EXPORTER_LIB} ${OPENTELEMETRY_TRACE_LIB} ${OPENTELEMETRY_RESOURCE_LIB} ${OPENTELEMETRY_COMMON_LIB})

清理项目，然后重新生成