专栏首页闵开慧运行wordcount时显示Could not obtain block

运行wordcount时显示Could not obtain block

该文章接上面hadoop运行wordcount时卡住不动,接着下面

hadoop@ubuntu118:~/hadoop-1.0.2$ bin/hadoop dfsadmin -safemode leave

Warning: $HADOOP_HOME is deprecated.

Safe mode is OFF

hadoop@ubuntu118:~/hadoop-1.0.2$ bin/hadoop jar hadoop-examples-1.0.2.jar wordcount file fileout66

Warning: $HADOOP_HOME is deprecated.

d****hdfs://192.168.0.118:9000/user/hadoop/file

12/09/10 18:35:52 INFO input.FileInputFormat: Total input paths to process : 2

12/09/10 18:35:53 INFO util.NativeCodeLoader: Loaded the native-hadoop library

12/09/10 18:35:53 WARN snappy.LoadSnappy: Snappy native library not loaded

12/09/10 18:35:53 INFO mapred.JobClient: Running job: job_201209101835_0001

12/09/10 18:35:54 INFO mapred.JobClient:  map 0% reduce 0%

12/09/10 18:36:17 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_0, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:20 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_0, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:35 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_1, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:35 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_1, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:50 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000000_2, Status : FAILED

java.io.IOException: Could not obtain block: blk_-7132067374412134968_1005 file=/user/hadoop/file/file2

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:36:50 INFO mapred.JobClient: Task Id : attempt_201209101835_0001_m_000001_2, Status : FAILED

java.io.IOException: Could not obtain block: blk_-4697097477082743821_1006 file=/user/hadoop/file/file1

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2266)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2060)

        at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2221)

        at java.io.DataInputStream.read(DataInputStream.java:83)

        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)

        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:396)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

12/09/10 18:37:12 INFO mapred.JobClient: Job complete: job_201209101835_0001

12/09/10 18:37:12 INFO mapred.JobClient: Counters: 6

12/09/10 18:37:12 INFO mapred.JobClient:   Job Counters 

12/09/10 18:37:12 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=115005

12/09/10 18:37:12 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0

12/09/10 18:37:12 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0

12/09/10 18:37:12 INFO mapred.JobClient:     Launched map tasks=8

12/09/10 18:37:12 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0

12/09/10 18:37:12 INFO mapred.JobClient:     Failed map tasks=1

执行文件file里面文件权限问题,查看里面的文件后发现其中的文件没有执行权限,但修改后发现还是没有执行权限。无奈执行重新上传文件上去hdfs,之后运行正常。

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • Windows中Eclipse中Hadoop2.6.0配置

    Windows10(64位)中Eclipse Luna Service Release 2 (4.4.2 64位)中Hadoop2.6.0配置 1 系统配置  ...

    闵开慧
  • Hmaster无法启动

    启动hbase时问题列表: 1 查看hbase-hadoop-master-ubuntu118.log,发现其中的错误为 2012-09-02 22:59:58...

    闵开慧
  • ERROR: org.apache.hadoop.hbase.MasterNotRunnin...

     ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times 这是为什么...

    闵开慧
  • Sqoop导入数据时异常java.net.ConnectException: Connection refused

    java.net.ConnectException: Call From node4/192.168.179.143 to node4:8032 failed ...

    时间静止不是简史
  • hadoop2.6分布式部署时 livenodes等于1的原因

    1.问题描述 在进行hadoop2.x版本的hdfs分布式部署时,遇到了一个奇怪的问题: 使用start-dfs.sh命令启动dfs之后,所有的datanode...

    老白
  • Amabri hive权限设置

    DataScience
  • 0524-6.1-如何使用Cloudera Manager启用HDFS的HA

    在HDFS集群中NameNode存在单点故障(SPOF),对于只有一个NameNode的集群,如果NameNode机器出现意外,将导致整个集群无法使用。为了解决...

    Fayson
  • java客户端无法上传文件到hdfs

    学些hadoop。遇到这个问题,查找网上好多资料,一般都是说namenode和datanode不同步导致的,或者防火墙没开50010端口,或者nameNode和...

    frontoldman
  • HBase案例 | 20000个分区导致HBase集群宕机事故处理

    这是几个月前遇到的一次HBase集群宕机事件,今天重新整理下事故分析报告。概况的说是业务方的一个10节点HBase集群支撑百TB级别的数据量,集群region数...

    大数据技术架构
  • 用java程序把本地的一个文件拷贝到hdfs上并显示进度

    把程序打成jar包放到Linux上 转到目录下执行命令 hadoop jar mapreducer.jar /home/c...

    用户5166556

扫码关注云+社区

领取腾讯云代金券