我创建并配置了小型hadoop集群(1主1从),并安装了spark。
我使用了预置的spark-1.6.0-bin-hadoop2.6
和hadoop-2.7
。
当我运行示例时
$ ./bin/spark-submit --class org.apache.spark.examples.SparkPi \
--master yarn \
--deploy-mode cluster \
--driver-memory 4g \
--executor-memory 2g \
--executor-cores 1 \
--queue thequeue \
lib/spark-examples*.jar \
10
从hduser (拥有/usr/local/hadoop
的用户)开始,一切正常,但是当我试图从同一个用户运行bin/spark-shell
时,出现了以下错误:
Database Class Loader started - derby.database.classpath='' 16/03/02 16:55:32 ERROR Datastore.Schema:
Failed initialising database. Cannot get a connection, pool error Could not create a validated object, cause:
A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
org.datanucleus.exceptions.NucleusDataStoreException: Cannot get a connection, pool error Could not create a validated object, cause:
A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
但是,当我从拥有/usr/local/spark
的user:user
运行bin/spark-shell
时,它工作得很好。
我有两个问题:
首先,为什么会发生这个错误?
第二,处理它的最好方法是什么?或者让一个用户同时拥有spark和hadoop?或者这是一种糟糕的做法?
https://stackoverflow.com/questions/35752979
复制相似问题