如何修复SparkUI执行器，java.io.FileNotFoundException
如何修复SparkUI执行器，java.io.FileNotFoundException
提问于 2019-04-23 15:45:53
我已经使用Apache Spark部署了Spring引导服务器，一切都很稳定。但http://X.X.X.X:4040/executors/ SparkUI executors终结点引发java.io.FileNotFoundException，找不到/opt/x/x!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar。我查过inner jar了。这个问题只发生在Linux上，on Windows它工作正常。
2019-04-23 07:01:24,038 WARN [org.spark_project.jetty.servlet.ServletHandler] [SparkUI-36] -
org.spark_project.jetty.servlet.ServletHolder$1: org.glassfish.jersey.server.internal.scanning.ResourceFinderException: java.io.FileNotFound
Exception: /opt/x/x.jar!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar (No such file or directory)
at org.spark_project.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:594)
at org.spark_project.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:658)
at org.spark_project.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:496)
at org.spark_project.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:788)
at org.spark_project.jetty.servlet.ServletHolder.prepare(ServletHolder.java:773)
at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:578)
at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.spark_project.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461)
at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.spark_project.jetty.server.Server.handle(Server.java:524)
at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:319)
at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
at org.spark_project.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.spark_project.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.spark_project.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.spark_project.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.spark_project.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.glassfish.jersey.server.internal.scanning.ResourceFinderException: java.io.FileNotFoundException: /opt/x/x.jar!/BOOT-INF/lib/spark-core_2.11-2.2.0.jar (No such file or directory)
at org.glassfish.jersey.server.internal.scanning.JarZipSchemeResourceFinderFactory.create(JarZipSchemeResourceFinderFactory.java:90)回答 1
Stack Overflow用户
回答已采纳
发布于 2019-07-28 17:19:41
问题的原因是Jersey中的一个限制-它不能处理嵌套的JAR文件。您需要将Spring Boot配置为在应用程序启动时自动解压任何包含spark-core资源的JAR。在我的例子中(我使用的是spark-core_2.12)，解决方案是在pom.xml文件中添加以下部分：
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<requiresUnpack>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
</dependency>
</requiresUnpack>
</configuration>
</plugin>页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/55806490
复制相关文章
相似问题
腾讯云开发者
