文章/答案/技术大牛

发布

社区首页 >问答首页 >包org.apache.hadoop.conf不存在

问包org.apache.hadoop.conf不存在
EN

Stack Overflow用户

提问于 2015-11-27 02:45:44

回答 2查看 26.8K关注 0票数 8

我刚开始编译Hadoop代码，我知道这个问题已经被多次回答了。但是，我仍然发现很难找到正确的类路径来使WordCount.java代码进行编译。我正在运行一个CentOS版本7。

我的Hadoop版本是：

Hadoop 2.7.1 Subversion   
https://git-wip-us.apache.org/repos/asf/hadoop.git -r 15ecc87ccf4a0228f35af08fc56de536e6ce657a   
Compiled by jenkins on 2015-06-29T06:04Z  
Compiled with protoc 2.5.0  
From source with checksum fc0a1a23fc1868e4d5ee7fa2b28a58a  
This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-2.7.1.jar

当我做回波$PATH时，我得到了以下信息：

/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/lib/jvm/jre-1.7.0-openjdk/:/usr/local/hadoop/bin:/usr/local/hadoop/sbin:/home/hduser/.local/bin:/home/hduser/bin:/usr/lib/jvm/jre-1.7.0-openjdk/:/usr/local/hadoop/bin:/usr/local/hadoop/sbin

我正在运行的Java版本是：

java version "1.7.0_91"
OpenJDK Runtime Environment (rhel-2.6.2.1.el7_1-x86_64 u91-b00)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)

我在vi .bashrc中为Java和Hadoop设置了所有环境变量，如下所示，jps通过了这些变量。Java也使用其他代码进行编译，但我认为我弄错了Hadoop的类路径。

Java变量

export JAVA_HOME=/usr/lib/jvm/jre-1.7.0-openjdk/
export PATH=$PATH:$JAVA_HOME

Hadoop变量

export HADOOP_INSTALL=/usr/local/hadoop 
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native

当我试图在工作目录中编译WordCount.java时，我会得到以下错误：

WordCount.java:5: error: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
WordCount.java:6: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.*;
^
WordCount.java:7: error: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
WordCount.java:8: error: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
WordCount.java:9: error: package org.apache.hadoop.mapreduce does not exist
import org.apache.hadoop.mapreduce.*;
^
WordCount.java:10: error: package org.apache.hadoop.mapreduce.lib.input does not exist
import org.apache.hadoop.mapreduce.lib.input.*;
^
WordCount.java:11: error: package org.apache.hadoop.mapreduce.lib.output does not exist
import org.apache.hadoop.mapreduce.lib.output.*;
^
WordCount.java:12: error: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.*;
^
WordCount.java:14: error: cannot find symbol
public class WordCount extends Configured implements Tool {
                               ^
  symbol: class Configured
WordCount.java:14: error: cannot find symbol
public class WordCount extends Configured implements Tool {
                                                     ^
  symbol: class Tool
WordCount.java:47: error: cannot find symbol
    public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
                                    ^
  symbol:   class Mapper
  location: class WordCount
WordCount.java:47: error: cannot find symbol
    public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
                                           ^
  symbol:   class LongWritable
  location: class WordCount
WordCount.java:47: error: cannot find symbol
    public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
                                                         ^
  symbol:   class Text
  location: class WordCount
WordCount.java:47: error: cannot find symbol
    public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
                                                               ^
  symbol:   class Text
  location: class WordCount
WordCount.java:47: error: cannot find symbol
    public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
                                                                     ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:48: error: cannot find symbol
    private final static IntWritable one = new IntWritable(1);
                         ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:49: error: cannot find symbol
    private Text word = new Text();
            ^
  symbol:   class Text
  location: class Map
WordCount.java:52: error: cannot find symbol
    public void map(LongWritable key, Text value,
                    ^
  symbol:   class LongWritable
  location: class Map
WordCount.java:52: error: cannot find symbol
    public void map(LongWritable key, Text value,
                                      ^
  symbol:   class Text
  location: class Map
WordCount.java:53: error: package Mapper does not exist
            Mapper.Context context) throws IOException, InterruptedException {
                  ^
WordCount.java:63: error: cannot find symbol
    public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
                                       ^
  symbol:   class Reducer
  location: class WordCount
WordCount.java:63: error: cannot find symbol
    public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
                                               ^
  symbol:   class Text
  location: class WordCount
WordCount.java:63: error: cannot find symbol
    public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
                                                     ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:63: error: cannot find symbol
    public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
                                                                  ^
  symbol:   class Text
  location: class WordCount
WordCount.java:63: error: cannot find symbol
    public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
                                                                        ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:66: error: cannot find symbol
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
                       ^
  symbol:   class Text
  location: class Reduce
WordCount.java:66: error: cannot find symbol
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
                                          ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:66: error: cannot find symbol
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException {
                                                               ^
  symbol:   class Context
  location: class Reduce
WordCount.java:17: error: cannot find symbol
    int res = ToolRunner.run(new WordCount(), args);
              ^
  symbol:   variable ToolRunner
  location: class WordCount
WordCount.java:22: error: cannot find symbol
    Path inputPath = new Path(args[0]);
    ^
  symbol:   class Path
  location: class WordCount
WordCount.java:22: error: cannot find symbol
    Path inputPath = new Path(args[0]);
                         ^
  symbol:   class Path
  location: class WordCount
WordCount.java:23: error: cannot find symbol
    Path outputPath = new Path(args[1]);
    ^
  symbol:   class Path
  location: class WordCount
WordCount.java:23: error: cannot find symbol
    Path outputPath = new Path(args[1]);
                          ^
  symbol:   class Path
  location: class WordCount
WordCount.java:25: error: cannot find symbol
    Configuration conf = getConf();
    ^
  symbol:   class Configuration
  location: class WordCount
WordCount.java:25: error: cannot find symbol
    Configuration conf = getConf();
                         ^
  symbol:   method getConf()
  location: class WordCount
WordCount.java:26: error: cannot find symbol
    Job job = new Job(conf, this.getClass().toString());
    ^
  symbol:   class Job
  location: class WordCount
WordCount.java:26: error: cannot find symbol
    Job job = new Job(conf, this.getClass().toString());
                  ^
  symbol:   class Job
  location: class WordCount
WordCount.java:26: error: cannot find symbol
    Job job = new Job(conf, this.getClass().toString());
                                ^
  symbol: method getClass()
WordCount.java:28: error: cannot find symbol
    FileInputFormat.setInputPaths(job, inputPath);
    ^
  symbol:   variable FileInputFormat
  location: class WordCount
WordCount.java:29: error: cannot find symbol
    FileOutputFormat.setOutputPath(job, outputPath);
    ^
  symbol:   variable FileOutputFormat
  location: class WordCount
WordCount.java:33: error: cannot find symbol
    job.setInputFormatClass(TextInputFormat.class);
                            ^
  symbol:   class TextInputFormat
  location: class WordCount
WordCount.java:34: error: cannot find symbol
    job.setOutputFormatClass(TextOutputFormat.class);
                             ^
  symbol:   class TextOutputFormat
  location: class WordCount
WordCount.java:35: error: cannot find symbol
    job.setMapOutputKeyClass(Text.class);
                             ^
  symbol:   class Text
  location: class WordCount
WordCount.java:36: error: cannot find symbol
    job.setMapOutputValueClass(IntWritable.class);
                               ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:37: error: cannot find symbol
    job.setOutputKeyClass(Text.class);
                          ^
  symbol:   class Text
  location: class WordCount
WordCount.java:38: error: cannot find symbol
    job.setOutputValueClass(IntWritable.class);
                            ^
  symbol:   class IntWritable
  location: class WordCount
WordCount.java:48: error: cannot find symbol
    private final static IntWritable one = new IntWritable(1);
                                               ^
  symbol:   class IntWritable
  location: class Map
WordCount.java:49: error: cannot find symbol
    private Text word = new Text();
                            ^
  symbol:   class Text
  location: class Map
WordCount.java:51: error: method does not override or implement a method from a supertype
    @Override
    ^
WordCount.java:68: error: cannot find symbol
    for (IntWritable value : values) {
         ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:72: error: cannot find symbol
    context.write(key, new IntWritable(sum));
                           ^
  symbol:   class IntWritable
  location: class Reduce
WordCount.java:65: error: method does not override or implement a method from a supertype
    @Override
    ^
52 errors

有人能给我指明正确的方向吗？谢谢。

linux

git

hadoop

jenkins

java

回答 2

Stack Overflow用户

发布于 2017-05-27 13:47:25

您可以像下面这样运行它：

javac WordCount.java -cp $(hadoop classpath)

如果出现以下错误，请执行以下操作：

zsh:未找到命令: hadoop javac：-cp需要一个参数

您可以尝试找到hadoop的确切位置，如下所示，就像我使用brew在Mac中安装的那样(在Linux中会有所不同)：

javac WordCount.java -cp $(/usr/local/Cellar/hadoop/2.8.0/bin/hadoop classpath)

hadoop类路径为编译器提供了正确编译所需的所有路径，您应该会看到目录中出现了一个结果WordCount.class。

票数 11

Stack Overflow用户

发布于 2015-11-27 04:12:20

我也有Hadoop2.7.1，我能够编译WordCount。

所有丢失的包都来自hadoop-common和hadoop-core jars。

您需要在您的pom.xml中包含以下Maven依赖项(我只有下面的依赖项，它为我编译)：

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.7.1</version>
        </dependency>
    
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.7.1</version>
        </dependency>
    
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.2.1</version>
        </dependency>
    </dependencies>

票数 1

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/33949589

复制

相似问题

问包org.apache.hadoop.conf不存在
EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问包org.apache.hadoop.conf不存在EN

回答 2

Stack Overflow用户

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问包org.apache.hadoop.conf不存在
EN