专栏首页牛肉圆粉不加葱ResourceManager HA无法连接Spark TrackUi现象解决方案

ResourceManager HA无法连接Spark TrackUi现象解决方案

现象

在对ResourceManager做了基于Zookeeper的HA后, 在YARN集群上执行Spark application后, 打开Spark Application Tracking UI显示以下错误:

HTTP ERROR 500

Problem accessing /proxy/application_1405708541410_0001/. Reason:

    Connection refused

Caused by:

java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at java.net.Socket.connect(Socket.java:538)
    at java.net.Socket.<init>(Socket.java:434)
    at java.net.Socket.<init>(Socket.java:286)
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
    at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
    at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
    at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:187)
    at org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:344)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:66)
    at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
    at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
    at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:84)
    at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1192)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

解决方案

  • 在Active ResourceManager和Standby ResourceManager及所有NodeManager上的yarn-site.xml上添加yarn.resourcemanager.webapp.address属性, 并重启相应的ResourceManager及NodeManager

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • Spark Streaming Failed to read checkpoint from directory ...现象解决方案及原因

    使用spark-submit提交一个Spark Streaming Application至yarn集群, 报错

    codingforfun
  • 配置hadoop集群namenode的hostname千万不要包含下划线

    在部署hadoop集群时,core-site.xml中的fs.defaultFS项的value不可包含下划线,否则会报以下错误

    codingforfun
  • Spark Shuffle 模块③ - Sort Based Shuffle writeSpark Shuffle 模块③ - Sort Based Shuffle write

    自 Spark 1.2 起,Sort Based Shuffle 替代 Hash Based Shuffle 成为 Spark 默认的 Shuffle 策略。

    codingforfun
  • 由一次线上故障来理解下TCP三握、四挥; Java堆栈分析到源码的探秘

    该服务主要是提供对外的代理接口,大部分接口都会调用第三方接口,获取数据后做聚合处理后,提供给客户端使用。

    黄泽杰
  • 由一次线上故障来理解下TCP三握、四挥 & Java堆栈分析到源码的探秘

    该服务主要是提供对外的代理接口,大部分接口都会调用第三方接口,获取数据后做聚合处理后,提供给客户端使用。

    云爬虫技术研究笔记
  • springboot2.0配置多数据源出错HikariPool-1 - jdbcUrl is required with driverClassName.

    版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。

    suveng
  • ABAP和Java SpringBoot的单元测试

    在ABAP类里,本地类(Local Class)里用关键字FOR TESTING声明过的方法,

    Jerry Wang
  • ABAP和Java SpringBoot的单元测试

    在ABAP类里,本地类(Local Class)里用关键字FOR TESTING声明过的方法,

    Jerry Wang
  • 聊聊hikari与tomcat jdbc pool的fail fast

    本文主要研究在中途数据库挂的情况下,hikari与tomcat jdbc pool的fail fast情况。

    codecraft
  • 2019-03-17 生产系统cup100%负载问题解决

    Arthas支持JDK 6+,支持Linux/Mac/Windows,采用命令行交互模式,同时提供丰富的 Tab 自动补全功能,进一步方便进行问题的定位和诊断。

    Albert陈凯

扫码关注云+社区

领取腾讯云代金券