前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Spark之Ubuntu18上安装Spark单机环境

Spark之Ubuntu18上安装Spark单机环境

作者头像
克虏伯
发布2019-04-15 09:56:09
1.1K0
发布2019-04-15 09:56:09
举报

    安装JDK,步骤略,

List-1

代码语言:javascript
复制
mjduan@mjduan-ubuntu:~$ java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

    安装Scala,下载地址

List-2

代码语言:javascript
复制
...

#scala
export SCALA_HOME=/opt/software/tool/scala2.12
export PATH=$SCALA_HOME/bin:$PATH

List-3

代码语言:javascript
复制
mjduan@mjduan-ubuntu:~$ source ~/.bashrc 
mjduan@mjduan-ubuntu:~$ scala -version
Scala code runner version 2.12.8 -- Copyright 2002-2018, LAMP/EPFL and Lightbend, Inc.

    安装Spark,下载地址

List-4

代码语言:javascript
复制
mjduan@mjduan-ubuntu:~$ tail -f ~/.bashrc 
......

#scala
export SCALA_HOME=/opt/software/tool/scala2.12
export PATH=$SCALA_HOME/bin:$PATH

#spark
export SPARK_HOME=/opt/software/tool/spark
export PATH=$SPARK_HOME/bin:$PATH

List-5  执行spark-shell可以看到命令行

代码语言:javascript
复制
mjduan@mjduan-ubuntu:~$ source ~/.bashrc 
mjduan@mjduan-ubuntu:~$ spark-shell 
2019-03-08 18:02:36 WARN  Utils:66 - Your hostname, mjduan-ubuntu resolves to a loopback address: 127.0.1.1; using 192.168.43.214 instead (on interface wlp2s0)
2019-03-08 18:02:36 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address
2019-03-08 18:02:37 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://192.168.43.214:4040
Spark context available as 'sc' (master = local[*], app id = local-1552039363367).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.4.0
      /_/
         
Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111)
Type in expressions to have them evaluated.
Type :help for more information.

scala> 

     在浏览器中输入http://192.168.43.214:4040就可看到UI界面了。

(adsbygoogle = window.adsbygoogle || []).push({});

本文参与 腾讯云自媒体同步曝光计划,分享自作者个人站点/博客。
如有侵权请联系 cloudcommunity@tencent.com 删除

本文分享自 作者个人站点/博客 前往查看

如有侵权,请联系 cloudcommunity@tencent.com 删除。

本文参与 腾讯云自媒体同步曝光计划  ,欢迎热爱写作的你一起参与!

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档