前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >copyToLocalFile报错:(null) entry in command string: null chmod 0644

copyToLocalFile报错:(null) entry in command string: null chmod 0644

作者头像
程裕强
发布2022-05-06 20:49:21
8820
发布2022-05-06 20:49:21
举报

1、测试程序,从HDFS上下载文件到本地

代码语言:javascript
复制
package com.cntaiping.tpa.storage;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import java.io.IOException;
import java.net.URI;
public class GetFileDemo {

    public static void main(String[] args) throws IOException{
        String hdfsPath="hdfs://192.168.1.180:8020/a/b.txt";
        String localPath="D:/copy.txt";
        Configuration conf = new Configuration();
        FileSystem fs = FileSystem.get(URI.create(hdfsPath), conf);
        Path hdfs_path = new Path(hdfsPath);
        Path local_path = new Path(localPath);
        fs.copyToLocalFile(hdfs_path, local_path);
        fs.close();

    }

}

2、报错

(null) entry in command string: null chmod 0644

代码语言:javascript
复制
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.io.IOException: (null) entry in command string: null chmod 0644 D:\copy.txt
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:770)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:866)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:849)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:733)
    at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:225)
    at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209)
    at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307)
    at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:296)
    at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:328)
    at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:398)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461)
    at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892)
    at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:365)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2034)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:2003)
    at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1979)
    at com.cntaiping.tpa.storage.GetFileDemo.main(GetFileDemo.java:16)

3、解决办法

代码语言:javascript
复制
        fs.copyToLocalFile(false,hdfs_path, local_path,true);

说明: 第一个false参数表示不删除源文件,第4个true参数表示使用本地原文件系统,因为这个Demo程序是在Windows系统下运行的。

4、原因分析

查看官方API http://hadoop.apache.org/docs/r2.7.6/api/

代码语言:javascript
复制
public void copyToLocalFile(boolean delSrc,
                            Path src,
                            Path dst,
                            boolean useRawLocalFileSystem)
                     throws IOException
The src file is under FS, and the dst is on the local disk. Copy it from FS control to the local dst name. delSrc indicates if the src will be removed or not. useRawLocalFileSystem indicates whether to use RawLocalFileSystem as local file system or not. RawLocalFileSystem is non crc file system.So, It will not create any crc files at local.
Parameters:
delSrc - whether to delete the src
src - path
dst - path
useRawLocalFileSystem - whether to use RawLocalFileSystem as local file system or not.
Throws:
IOException - - if any IO error
本文参与 腾讯云自媒体分享计划,分享自作者个人站点/博客。
原始发表:2018-08-21,如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体分享计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 1、测试程序,从HDFS上下载文件到本地
  • 2、报错
  • 3、解决办法
  • 4、原因分析
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档