版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:[https://blog.csdn.net/boling\_cavalry/article/details/102765992](https://blog.csdn.net/boling_cavalry/article/details/102765992)
请参考《Mac部署hadoop3(伪分布式)》一文,在Mac上事先部署好hadoop3
brew install scala
export SCALA_HOME=/usr/local/Cellar/scala/2.13.0
export PATH=$PATH:$SCALA_HOME/bin
base) zhaoqindeMBP:~ zhaoqin$ scala -version
Scala code runner version 2.13.0 -- Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc.
export SPARK_HOME=/usr/local/spark
export PATH=$PATH:$SPARK_HOME/bin
export SCALA_HOME=/usr/local/Cellar/scala/2.13.0
export SPARK_MASTER_IP=localhost
export SPARK_WORKER_MEMORY=2G
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(base) zhaoqindeMBP:~ zhaoqin$ spark-shell
19/10/27 13:33:51 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://zhaoqindembp:4040
Spark context available as 'sc' (master = local[*], app id = local-1572154437623).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.4
/_/
Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_211)
Type in expressions to have them evaluated.
Type :help for more information.
scala>
至此,Mac机器上的hadoop和spark都运行起来了,希望本文能给您带来一些参考。