Ambari集成Elasticsearch 5.x

1、资源URL

https://github.com/Jaraxal/ambari-elasticsearch-service 下载得到ambari-elasticsearch-service-master.zip压缩包

2、首先添加服务的stack

(1)进入ambari源码ambari-server/resources/stacks/HDP/2.4/services目录

[root@master services]# pwd
/var/lib/ambari-server/resources/stacks/HDP/2.4/services

[root@master services]# ls

ACCUMULO  FALCON  HBASE  HIVE   KERBEROS  MAHOUT  PIG     RANGER_KMS  SPARK  stack_advisor.py   stack_advisor.pyo  TEZ   ZOOKEEPER

ATLAS     FLUME   HDFS   KAFKA  KNOX      OOZIE   RANGER  SLIDER      SQOOP  stack_advisor.pyc  STORM              YARN

(2)创建ELASTICSEARCH目录并进入该目录

[root@master services]# mkdir ELASTICSEARCH
[root@master services]# cd ELASTICSEARCH/

(3)编辑metainfo.xml

[root@master ELASTICSEARCH]#vi metainfo.xml
[root@master ELASTICSEARCH]#cat metainfo.xml
<?xml version="1.0"?>
    <metainfo>
      <schemaVersion>2.0</schemaVersion>
      <services>
        <service>
          <name>ELASTICSEARCH</name>
          <version>5.1.1</version>
          <extends>common-services/ELASTICSEARCH/5.1.1</extends>
        </service>
      </services>
    </metainfo>

3、ambari-server/src/main/resources/common-services

(1)进入ambari-server/src/main/resources/common-services目录

[root@master common-services]# pwd
/opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services

[root@master common-services]# ls
ACCUMULO        ATLAS          FLUME    HBASE  KAFKA     LOGSEARCH  PIG     RANGER_KMS  SPARK2  TEZ       ZOOKEEPER

AMBARI_INFRA    ELASTICSEARCH  GANGLIA  HDFS   KERBEROS  MAHOUT     PXF     SLIDER      SQOOP   YARN

AMBARI_METRICS  FALCON         HAWQ     HIVE   KNOX      OOZIE      RANGER  SPARK       STORM   ZEPPELIN

(2)创建ES版本目录

[root@master common-services]# mkdir -p ELASTICSEARCH/5.1.1/

[root@vnode1 common-services]# cd 

(3)解压ambari-elasticsearch-service-master.zip

[root@master ~]# unzip ambari-elasticsearch-service-master.zip

(4)编辑ambari-elasticsearch-service-master中的metainfo.xml文件

[root@master ~]# cd ambari-elasticsearch-service-master    

[root@master ambari-elasticsearch-service-master]# vi metainfo.xml
[root@vnode0 5.1.1]# vi metainfo.xml

<?xml version="1.0"?>
<metainfo>
    <schemaVersion>2.0</schemaVersion>
    <services>
        <service>
            <name>ELASTICSEARCH</name>
            <displayName>ElasticSearch</displayName>
            <comment>ElasticSearch service</comment>
            <version>5.1.1</version>
            <components>
                <component>
                    <name>ELASTICSEARCH</name>
                    <displayName>ElasticSearch</displayName>
                    <category>MASTER</category>
                    <cardinality>1</cardinality>
                    <commandScript>
                        <script>scripts/master.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                </component>
                <component>
                    <name>ELASTICSEARCH_NODE</name>
                    <displayName>ElasticSearchNode</displayName>
                    <category>SLAVE</category>
                    <cardinality>ALL</cardinality>
                    <!--自动部署,仅在cardinality满足的情况下,界面上就不问,在这台主机,是否安装这个组件了-->
                    <auto-deploy> 
                        <enabled>false</enabled>
                    </auto-deploy>
                    <commandScript>
                        <script>scripts/slave.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                    <!--<configFiles>
                    <configFile>
                        <type>env</type>
                        <fileName>elasticsearch-env.xml</fileName>
                        <dictionaryName>elasticsearch-env</dictionaryName>
                     </configFile>
                    </configFiles>-->
                </component>
            </components>
            <osSpecifics>
                <osSpecific>
                  <osFamily>any</osFamily>
                  <packages>
                    <package>
                      <name>elasticsearch</name>
                    </package>
                  </packages>
                </osSpecific>
            </osSpecifics>
            <requiredServices>
                <service>GANGLIA</service> <!--依赖服务,安装的时候限制,没有不能继续安装-->
            </requiredServices>
            <configuration-dependencies>
              <config-type>elasticsearch-env</config-type>
            </configuration-dependencies>
            <monitoringService>false</monitoringService> <!--如果是监控服务的话,每个组件重启,他都要重启-->
        </service>
    </services>
</metainfo>

(5)复制脚本 将https://github.com/Jaraxal/ambari-elasticsearch-service下载得到ambari-elasticsearch-service-master.zip压缩包的配置文件复制到ambari对应目录下。

[root@master ambari-elasticsearch-service-master]# cp -r * /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

(6)查看

[root@master ambari-elasticsearch-service-master]# cd /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

[root@master 5.1.1]# ls

configuration  LICENSE  metainfo.xml  package  quicklinks  README.md

[root@master 5.1.1]# 

(7)设置ES下载URL

[root@master 5.1.1]# cd package/scripts/

[root@master scripts]# ls

elastic_common.py  master.py  params.py  status_params.py

[root@master scripts]# vi params.py 

elastic_download = 'http://192.168.1.180/hdp/elasticsearch-5.1.1.tar.gz'
[root@master 2.4.3.0]# vi HDP-2.4.3.0-227.xml

<?xml version="1.0"?>

<repository-version xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="version_definition.xsd">

  <release>

    <type>STANDARD</type>

    <stack-id>HDP-2.4</stack-id>

    <version>2.4.3.0</version>

    <build>227</build>

    <compatible-with>2.4.[0-9]+</compatible-with>

    <release-notes>http://example.com</release-notes>

    <display>HDP-2.4.3.0-227</display>

    <package-version>2_4_3_0_*</package-version>

  </release>

  <manifest>

    <service id="KNOX-060" name="KNOX" version="0.6.0"/>

    <service id="SLIDER-0800" name="SLIDER" version="0.80.0"/>

    <service id="FALCON-061" name="FALCON" version="0.6.1"/>

    <service id="MAHOUT-090" name="MAHOUT" version="0.9.0"/>

    <service id="HBASE-112" name="HBASE" version="1.1.2"/>

    <service id="FLUME-152" name="FLUME" version="1.5.2"/>

    <service id="HIVE-121000" name="HIVE" version="1.2.1000"/>

    <service id="TEZ-070" name="TEZ" version="0.7.0"/>

    <service id="OOZIE-420" name="OOZIE" version="4.2.0"/>

    <service id="HDFS-271" name="HDFS" version="2.7.1"/>

    <service id="YARN-271" name="YARN" version="2.7.1"/>

    <service id="MAPREDUCE2-271" name="MAPREDUCE2" version="2.7.1"/>

    <service id="ZOOKEEPER-346" name="ZOOKEEPER" version="3.4.6"/>

    <service id="KAFKA-090" name="KAFKA" version="0.9.0"/>

    <service id="SQOOP-146" name="SQOOP" version="1.4.6"/>

    <service id="ATLAS-050" name="ATLAS" version="0.5.0"/>

    <service id="SPARK-162" name="SPARK" version="1.6.2"/>

    <service id="STORM-0100" name="STORM" version="0.10.0"/>

    <service id="PIG-0150" name="PIG" version="0.15.0"/>

    <service id="RANGER-050" name="RANGER" version="0.5.0"/>

    <service id="RANGER_KMS-050" name="RANGER_KMS" version="0.5.0"/>

    <service id="ACCUMULO-170" name="ACCUMULO" version="1.7.0"/>

    <service id="ELASTICSEARCH-511" name="ELASTICSEARCH" version="5.1.1"/>

  </manifest>

  <available-services/>

  <repository-info>

    <os family="redhat6">

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/2.x/BUILDS/2.4.3.0-227</baseurl>

        <repoid>HDP-2.4</repoid>

        <reponame>HDP</reponame>

      </repo>

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7</baseurl>

        <repoid>HDP-UTILS-1.1.0.20</repoid>

        <reponame>HDP-UTILS</reponame>

      </repo>

    </os>

  </repository-info>

</repository-version>
[root@master 2.4.3.0]# createrepo --update .

Spawning worker 0 with 46 pkgs

Spawning worker 1 with 46 pkgs

Spawning worker 2 with 45 pkgs

Spawning worker 3 with 45 pkgs

Workers Finished

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

1 条评论
登录 后参与评论

相关文章

来自专栏代码散人

Vapor3初探——使用Fluent查询数据

Vapor升级到Vapor3的时间恰巧赶上WWDC,本来想从美国回来就开始写一些关于Vapor3的文章,但是WWDC信息量太大了,直到现在才转过来写一点关于Va...

793
来自专栏吴小龙同學

android之获取手机信息

android获取手机信息(号码,内存,CPU,分辨率,MAC,IP,SD卡,IMEI,经纬度,信号强度等等) 1 2 3 4 5 6 7 8 9 10 11 ...

3417
来自专栏WindCoder

进击微信小程序-Java版AES/CBC/PKCS7Padding加解密字符串

进击微信小程序,最初找到腾讯云给的解决方案,无奈后端是node.js,而且才一开始配上去还不能用,之后仔细看readme,摸到它开源的Wafer,本来以为找到终...

432
来自专栏salesforce零基础学习

salesforce 零基础学习(五十六)实现getById

本来想通过template封装DAO中的getById,结果template中无法选择$(object_name),所以此种想法打消了,直接封装成一个Helpe...

1815
来自专栏张高兴的博客

张高兴的 Windows 10 IoT 开发笔记:BH1750FVI 光照度传感器

2886
来自专栏码匠的流水账

聊聊HystrixMetricsPublisher

hystrix-core-1.5.12-sources.jar!/com/netflix/hystrix/strategy/metrics/HystrixMet...

451
来自专栏hbbliyong

使用线程新建WPF窗体(公用进度条窗体)

使用线程新建窗体 项目中需要一个公用的进度条窗体.大家知道在wpf中,有两个线程,一个是UI线程,另一个是监听线程(一直监听用户的输入).如果我们后台有阻塞UI...

37110
来自专栏码匠的流水账

聊聊spring cloud的HystrixCircuitBreakerConfiguration

本文主要研究一下spring cloud的HystrixCircuitBreakerConfiguration

472
来自专栏知识分享

用51单片机+11.0592的晶振,如何产生115200的波特率

原文地址http://homezzy.blog.hexun.com/29601793_d.html 以前自己也遇到过这种情况,模块115200波特率而且不能更改...

3179
来自专栏移动开发之家

Android图片加载开源库深度推荐,安利Fresco1、Universal-Image-Loader 2、Picasso 3、Glide 4、Fresco

</p> 这里没有广告,这里没有跑分,数据对比Google一下比比皆是,额···那我说什么好(ノಠ益ಠ)ノ彡┻━┻。 对啊,女朋友···呸呸呸,那么就来讲一...

462

扫码关注云+社区