1、执行脚本程序报如下所示的错误:
1 [hadoop@slaver1 script_hadoop]$ hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree /home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 /home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19
2 18/05/20 17:09:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
3 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C
4 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
5 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
6 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
7 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
8 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
9 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
10 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
11 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498)
12 at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207)
13 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
15 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16 at java.lang.reflect.Method.invoke(Method.java:606)
17 at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
18 [hadoop@slaver1 script_hadoop]$ hadoop jar web_click_mr_hive.jar com.bie.hive.mr.ClickStreamThree hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/output/2018-05-19 hdfs://slaver1:9000/home/hadoop/data_hadoop/weblog/preprocess/click_pv_out/2018-05-19
19 18/05/20 17:10:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20 Exception in thread "main" java.io.IOException: No FileSystem for scheme: C
21 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
22 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
23 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
24 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
25 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
26 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
27 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
28 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:498)
29 at com.bie.hive.mr.ClickStreamThree.main(ClickStreamThree.java:207)
30 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
31 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
32 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
33 at java.lang.reflect.Method.invoke(Method.java:606)
34 at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
35 [hadoop@slaver1 script_hadoop]$
2、猛一看,WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable,百度一搜这个错误,啥编译32/64位的问题,也有说改配置文件的问题,也许有的可能是这样改就解决问题了,而我想说,出现问题,看看报错原因,也许就可以很快的解决问题了。
然后呢,看下代码,是如下所示导致的错误,将下面的注释了,代码换成上面的就可以使用脚本程序执行代码了:
FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
//FileInputFormat.setInputPaths(job, new Path("C:/weblog/output"));
//FileOutputFormat.setOutputPath(job, new Path("c:/weblog/pageviews"));