前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >运行wordcount时显示Could not obtain block

运行wordcount时显示Could not obtain block

作者头像
闵开慧
发布2018-03-30 12:13:22
8290
发布2018-03-30 12:13:22
举报
文章被收录于专栏:闵开慧闵开慧

该文章接上面hadoop运行wordcount时卡住不动,接着下面

hadoop@ubuntu118:~/hadoop-1.0.2$ bin/hadoop dfsadmin -safemode leave

Warning: $HADOOP_HOME is deprecated.

Safe mode is OFF

hadoop@ubuntu118:~/hadoop-1.0.2$ bin/hadoop jar hadoop-examples-1.0.2.jar wordcount file fileout66

Warning: $HADOOP_HOME is deprecated.

d****hdfs://192.168.0.118:9000/user/hadoop/file

12/09/10 18:35:52 INFO input.FileInputFormat: Total input paths to process : 2

12/09/10 18:35:53 INFO util.NativeCodeLoader: Loaded the native-hadoop library

12/09/10 18:35:53 WARN snappy.LoadSnappy: Snappy native library not loaded

12/09/10 18:35:53 INFO mapred.JobClient: Running job: job_201209101835_0001

12/09/10 18:35:54 INFO mapred.JobClient:  map 0% reduce 0%

12/09/10 18:36:17 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_0, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:20 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_0, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:35 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_1, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:35 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_1, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:50 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_2, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:50 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_2, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:37:12 INFO mapred.JobClient: Job complete: job_201209101835_0001

12/09/10 18:37:12 INFO mapred.JobClient: Counters: 6

12/09/10 18:37:12 INFO mapred.JobClient:   Job Counters 

12/09/10 18:37:12 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=115005

12/09/10 18:37:12 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0

12/09/10 18:37:12 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0

12/09/10 18:37:12 INFO mapred.JobClient:     Launched map tasks=8

12/09/10 18:37:12 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0

12/09/10 18:37:12 INFO mapred.JobClient:     Failed map tasks=1

执行文件file里面文件权限问题,查看里面的文件后发现其中的文件没有执行权限,但修改后发现还是没有执行权限。无奈执行重新上传文件上去hdfs,之后运行正常。

本文参与 腾讯云自媒体分享计划,分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档