apache-使用org.apache.spark.unsafe.Platform java.nio.DirectByteBuffer(长、int)的非法反射访问来运行警告。
apache-使用org.apache.spark.unsafe.Platform java.nio.DirectByteBuffer(长、int)的非法反射访问来运行警告。
提问于 2022-01-11 00:42:52
我通过自制软件安装了apache火花。
brew install apache-spark然后跑
Spark-shell 并返回以下警告：
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/usr/local/Cellar/apache-spark/3.2.0/libexec/jars/spark-unsafe_2.12-3.2.0.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/01/10 18:21:03 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Spark context Web UI available at http://ip-192-168-1-176.ec2.internal:4040
Spark context available as 'sc' (master = local[*], app id = local-1641860464071).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 3.2.0
/_/
Using Scala version 2.12.15 (OpenJDK 64-Bit Server VM, Java 11.0.12)
Type in expressions to have them evaluated.
Type :help for more information.这是我的java版本。
openjdk 11.0.12 2021-07-20
OpenJDK Runtime Environment Homebrew (build 11.0.12+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.12+0, mixed mode)如何解决这些警告？
回答 1
Stack Overflow用户
发布于 2022-01-11 19:50:09
要消除这些错误，需要解决两个问题：
- Java11和Spark3.2一起使用时会产生这些错误。第一步是切换到Java8，直到这个问题得到解决。
- 使用brew获得了与您相同的错误，即使我使用的是Java8。而不是使用brew，转到https://www.apache.org/dyn/closer.lua/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.2.tgz的Apache网站并解压缩这个文件。若要设置PATH变量，以便键入“星火-外壳”，则需要进入您的.bash、.profile或.zshrc (无论您使用哪种)，并添加“导出火花_HOME=路径到火花文件夹”，然后将$SPARK_HOME添加到路径变量.
中。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/70660419
复制相关文章
相似问题
腾讯云开发者
