前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin

作者头像
Java架构师必看
发布2021-08-06 15:57:26
9000
发布2021-08-06 15:57:26
举报
文章被收录于专栏:Java架构师必看Java架构师必看

问题:

    每个人安装hadoop时,都可能会遇见这个问题“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable“,然后转手就去百度。也不知道网上怎么传的,都说是版本问题,各种编译就上来了,我就日了,这坑了多少人,几天的时间就这么被坑了。

正解:

    作为程序员,遇见问题我们第一时间应该是去看日志的,伸手党们习惯直接百度,然后一步一步走弯路。

    由于hadoop默认的日志级别较低,执行命令export HADOOP_ROOT_LOGGER=DEBUG,console改一下,会发现日志里面写的很清晰:

代码语言:javascript
复制
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
17/11/06 18:53:16 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
17/11/06 18:53:16 DEBUG security.Groups:  Creating new Groups object
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0)
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: java.library.path=/hadoop/hadoop-2.7.1/lib/native
17/11/06 18:53:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/06 18:53:16 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/11/06 18:53:16 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/11/06 18:53:16 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login commit
17/11/06 18:53:17 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: User entry: "root"
17/11/06 18:53:17 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
17/11/06 18:53:17 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)

    一眼就能看出问题的所在,glibc问题,人家需要的版本高,你应该升级一下。

使用命令:strings /lib64/libc.so.6 |grep GLIBC_

代码语言:javascript
复制
[root@hadoop02 sbin]# strings /lib64/libc.so.6 |grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE

发现自己的版本低,解决办法就是升级,引用别人的劳动成果。

     上一篇遇见问题,参考下面解决了。

    http://www.linuxdiyf.com/linux/23702.html

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档