WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin

Java架构师必看

发布于 2021-08-06 15:57:26

9050

发布于 2021-08-06 15:57:26

问题：

每个人安装hadoop时，都可能会遇见这个问题“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable“，然后转手就去百度。也不知道网上怎么传的，都说是版本问题，各种编译就上来了，我就日了，这坑了多少人，几天的时间就这么被坑了。

正解：

作为程序员，遇见问题我们第一时间应该是去看日志的，伸手党们习惯直接百度，然后一步一步走弯路。

由于hadoop默认的日志级别较低，执行命令export HADOOP_ROOT_LOGGER=DEBUG,console改一下，会发现日志里面写的很清晰：

17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
17/11/06 18:53:16 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
17/11/06 18:53:16 DEBUG security.Groups:  Creating new Groups object
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0)
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: java.library.path=/hadoop/hadoop-2.7.1/lib/native
17/11/06 18:53:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/06 18:53:16 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/11/06 18:53:16 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/11/06 18:53:16 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login commit
17/11/06 18:53:17 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: User entry: "root"
17/11/06 18:53:17 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
17/11/06 18:53:17 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)

一眼就能看出问题的所在，glibc问题，人家需要的版本高，你应该升级一下。

使用命令：strings /lib64/libc.so.6 |grep GLIBC_

[root@hadoop02 sbin]# strings /lib64/libc.so.6 |grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE

发现自己的版本低，解决办法就是升级，引用别人的劳动成果。

上一篇遇见问题，参考下面解决了。

http://www.linuxdiyf.com/linux/23702.html

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

如有侵权请联系 cloudcommunity@tencent.com 删除

hadoop