Livy创建会话死亡
Livy创建会话死亡
提问于 2020-05-04 00:36:32
我将一个包(在spark-default.conf)中)添加到我的see配置中(但当我用livy创建一个新会话时，它会给我带来一个问题(参见下面的错误)以及会话和死亡。ps:当我移除这个包的时候，一切都很好。
20/05/04 00:17:35 WARN RSCClient: Error stopping RPC.
io.netty.util.concurrent.BlockingOperationException: DefaultChannelPromise@6d493840(uncancellable)
at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
...........
Exception in thread "Thread-32" java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)
........
at org.apache.livy.utils.LineBufferedStream$$anon$1.run(LineBufferedStream.scala:46)
20/05/04 00:17:36 WARN ContextLauncher: Child process exited with code 143.
20/05/04 00:17:36 ERROR SparkProcApp: job was killed by user
20/05/04 00:17:36 INFO InteractiveSession: Stopped InteractiveSession 0.
20/05/04 00:28:17 INFO InteractiveSessionManager: Deleting InteractiveSession 0 because it was inactive for more than 3600000.0 ms.
20/05/04 00:28:17 INFO InteractiveSessionManager: Deleting session 0
20/05/04 00:28:17 INFO InteractiveSession: Stopping InteractiveSession 0...
20/05/04 00:28:17 INFO InteractiveSession: Stopped InteractiveSession 0.
20/05/04 00:28:17 INFO InteractiveSessionManager: Deleted session 0 我使用: cloudera hdp2.6.5：
- 火花2.3
- livy 0.7.0
- Hadoop 2.7
- 自由无监督(https://github.com/unsupervise/spark-tss)
步骤：
- 长春藤=> livy.spark.master纱簇
- 火花-默认的conf => spark.jars.repositories https://dl.bintray.com/unsupervise/maven/
- 星星之火-defaultconf => spark.jars.packages com.github.unvisor:spark:0.1.1
回答 1
Stack Overflow用户
发布于 2020-05-13 20:43:06
也请尝试添加以激发默认值。
spark.jars.repositories https://dl.bintray.com/unsupervise/maven/或者(为了确保您对spark-default.conf文件没有任何问题)，请尝试在提交火花作业时将这些信任信息包含到Livy中(参见Livy API)。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/61583469
复制相关文章
相似问题
腾讯云开发者
