Ambari集成Elasticsearch 5.x

1、资源URL

https://github.com/Jaraxal/ambari-elasticsearch-service 下载得到ambari-elasticsearch-service-master.zip压缩包

2、首先添加服务的stack

(1)进入ambari源码ambari-server/resources/stacks/HDP/2.4/services目录

[root@master services]# pwd
/var/lib/ambari-server/resources/stacks/HDP/2.4/services

[root@master services]# ls

ACCUMULO  FALCON  HBASE  HIVE   KERBEROS  MAHOUT  PIG     RANGER_KMS  SPARK  stack_advisor.py   stack_advisor.pyo  TEZ   ZOOKEEPER

ATLAS     FLUME   HDFS   KAFKA  KNOX      OOZIE   RANGER  SLIDER      SQOOP  stack_advisor.pyc  STORM              YARN

(2)创建ELASTICSEARCH目录并进入该目录

[root@master services]# mkdir ELASTICSEARCH
[root@master services]# cd ELASTICSEARCH/

(3)编辑metainfo.xml

[root@master ELASTICSEARCH]#vi metainfo.xml
[root@master ELASTICSEARCH]#cat metainfo.xml
<?xml version="1.0"?>
    <metainfo>
      <schemaVersion>2.0</schemaVersion>
      <services>
        <service>
          <name>ELASTICSEARCH</name>
          <version>5.1.1</version>
          <extends>common-services/ELASTICSEARCH/5.1.1</extends>
        </service>
      </services>
    </metainfo>

3、ambari-server/src/main/resources/common-services

(1)进入ambari-server/src/main/resources/common-services目录

[root@master common-services]# pwd
/opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services

[root@master common-services]# ls
ACCUMULO        ATLAS          FLUME    HBASE  KAFKA     LOGSEARCH  PIG     RANGER_KMS  SPARK2  TEZ       ZOOKEEPER

AMBARI_INFRA    ELASTICSEARCH  GANGLIA  HDFS   KERBEROS  MAHOUT     PXF     SLIDER      SQOOP   YARN

AMBARI_METRICS  FALCON         HAWQ     HIVE   KNOX      OOZIE      RANGER  SPARK       STORM   ZEPPELIN

(2)创建ES版本目录

[root@master common-services]# mkdir -p ELASTICSEARCH/5.1.1/

[root@vnode1 common-services]# cd 

(3)解压ambari-elasticsearch-service-master.zip

[root@master ~]# unzip ambari-elasticsearch-service-master.zip

(4)编辑ambari-elasticsearch-service-master中的metainfo.xml文件

[root@master ~]# cd ambari-elasticsearch-service-master    

[root@master ambari-elasticsearch-service-master]# vi metainfo.xml
[root@vnode0 5.1.1]# vi metainfo.xml

<?xml version="1.0"?>
<metainfo>
    <schemaVersion>2.0</schemaVersion>
    <services>
        <service>
            <name>ELASTICSEARCH</name>
            <displayName>ElasticSearch</displayName>
            <comment>ElasticSearch service</comment>
            <version>5.1.1</version>
            <components>
                <component>
                    <name>ELASTICSEARCH</name>
                    <displayName>ElasticSearch</displayName>
                    <category>MASTER</category>
                    <cardinality>1</cardinality>
                    <commandScript>
                        <script>scripts/master.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                </component>
                <component>
                    <name>ELASTICSEARCH_NODE</name>
                    <displayName>ElasticSearchNode</displayName>
                    <category>SLAVE</category>
                    <cardinality>ALL</cardinality>
                    <!--自动部署,仅在cardinality满足的情况下,界面上就不问,在这台主机,是否安装这个组件了-->
                    <auto-deploy> 
                        <enabled>false</enabled>
                    </auto-deploy>
                    <commandScript>
                        <script>scripts/slave.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                    <!--<configFiles>
                    <configFile>
                        <type>env</type>
                        <fileName>elasticsearch-env.xml</fileName>
                        <dictionaryName>elasticsearch-env</dictionaryName>
                     </configFile>
                    </configFiles>-->
                </component>
            </components>
            <osSpecifics>
                <osSpecific>
                  <osFamily>any</osFamily>
                  <packages>
                    <package>
                      <name>elasticsearch</name>
                    </package>
                  </packages>
                </osSpecific>
            </osSpecifics>
            <requiredServices>
                <service>GANGLIA</service> <!--依赖服务,安装的时候限制,没有不能继续安装-->
            </requiredServices>
            <configuration-dependencies>
              <config-type>elasticsearch-env</config-type>
            </configuration-dependencies>
            <monitoringService>false</monitoringService> <!--如果是监控服务的话,每个组件重启,他都要重启-->
        </service>
    </services>
</metainfo>

(5)复制脚本 将https://github.com/Jaraxal/ambari-elasticsearch-service下载得到ambari-elasticsearch-service-master.zip压缩包的配置文件复制到ambari对应目录下。

[root@master ambari-elasticsearch-service-master]# cp -r * /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

(6)查看

[root@master ambari-elasticsearch-service-master]# cd /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

[root@master 5.1.1]# ls

configuration  LICENSE  metainfo.xml  package  quicklinks  README.md

[root@master 5.1.1]# 

(7)设置ES下载URL

[root@master 5.1.1]# cd package/scripts/

[root@master scripts]# ls

elastic_common.py  master.py  params.py  status_params.py

[root@master scripts]# vi params.py 

elastic_download = 'http://192.168.1.180/hdp/elasticsearch-5.1.1.tar.gz'
[root@master 2.4.3.0]# vi HDP-2.4.3.0-227.xml

<?xml version="1.0"?>

<repository-version xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="version_definition.xsd">

  <release>

    <type>STANDARD</type>

    <stack-id>HDP-2.4</stack-id>

    <version>2.4.3.0</version>

    <build>227</build>

    <compatible-with>2.4.[0-9]+</compatible-with>

    <release-notes>http://example.com</release-notes>

    <display>HDP-2.4.3.0-227</display>

    <package-version>2_4_3_0_*</package-version>

  </release>

  <manifest>

    <service id="KNOX-060" name="KNOX" version="0.6.0"/>

    <service id="SLIDER-0800" name="SLIDER" version="0.80.0"/>

    <service id="FALCON-061" name="FALCON" version="0.6.1"/>

    <service id="MAHOUT-090" name="MAHOUT" version="0.9.0"/>

    <service id="HBASE-112" name="HBASE" version="1.1.2"/>

    <service id="FLUME-152" name="FLUME" version="1.5.2"/>

    <service id="HIVE-121000" name="HIVE" version="1.2.1000"/>

    <service id="TEZ-070" name="TEZ" version="0.7.0"/>

    <service id="OOZIE-420" name="OOZIE" version="4.2.0"/>

    <service id="HDFS-271" name="HDFS" version="2.7.1"/>

    <service id="YARN-271" name="YARN" version="2.7.1"/>

    <service id="MAPREDUCE2-271" name="MAPREDUCE2" version="2.7.1"/>

    <service id="ZOOKEEPER-346" name="ZOOKEEPER" version="3.4.6"/>

    <service id="KAFKA-090" name="KAFKA" version="0.9.0"/>

    <service id="SQOOP-146" name="SQOOP" version="1.4.6"/>

    <service id="ATLAS-050" name="ATLAS" version="0.5.0"/>

    <service id="SPARK-162" name="SPARK" version="1.6.2"/>

    <service id="STORM-0100" name="STORM" version="0.10.0"/>

    <service id="PIG-0150" name="PIG" version="0.15.0"/>

    <service id="RANGER-050" name="RANGER" version="0.5.0"/>

    <service id="RANGER_KMS-050" name="RANGER_KMS" version="0.5.0"/>

    <service id="ACCUMULO-170" name="ACCUMULO" version="1.7.0"/>

    <service id="ELASTICSEARCH-511" name="ELASTICSEARCH" version="5.1.1"/>

  </manifest>

  <available-services/>

  <repository-info>

    <os family="redhat6">

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/2.x/BUILDS/2.4.3.0-227</baseurl>

        <repoid>HDP-2.4</repoid>

        <reponame>HDP</reponame>

      </repo>

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7</baseurl>

        <repoid>HDP-UTILS-1.1.0.20</repoid>

        <reponame>HDP-UTILS</reponame>

      </repo>

    </os>

  </repository-info>

</repository-version>
[root@master 2.4.3.0]# createrepo --update .

Spawning worker 0 with 46 pkgs

Spawning worker 1 with 46 pkgs

Spawning worker 2 with 45 pkgs

Spawning worker 3 with 45 pkgs

Workers Finished

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏乐沙弥的世界

MHA 切换的2个异常(masterha_master_switch line 53)

        MHA 在测试手动故障转移和在线切换的过程中,碰到了2个比较诡异的问题,在使用IP地址调用的时候均无法测试成功,出现了Detected dead...

412
来自专栏生信技能树

GATK4的gvcf流程

得到了它们的bam文件,也是走的GATK流程,这里就不多说了。本教程首发于生信技能树VIP论坛:https://vip.biotrainee.com/d/423...

1713
来自专栏Hongten

python开发_tkinter_菜单选项中英文切换_菜单选项不可用操作_博主推荐

=================================================

1473
来自专栏bboysoul

linux服务器性能测试脚本serverreview-benchmark

欢迎关注Bboysoul的博客www.bboysoul.com Have Fun

853
来自专栏技术小黑屋

Jar Mismatch! Fix Your Dependencies

There was a requirement of my work. It requires me to integrated my current proj...

712
来自专栏散尽浮华

kvm虚拟化关闭虚拟网卡virbr0的方法

我们知道:kvm虚拟化环境安装好后,ifconfig会发现多了一个虚拟网卡virbr0 这是由于安装和启用了libvirt服务后生成的,libvirt在服务器(...

17511
来自专栏程序员笔记

Note_Motivation & Gamification

1946
来自专栏乐沙弥的世界

Percona XtraDB Cluster添加仲裁节点

Galera Arbitrator是Percona XtraDB集群的成员,用于投票,以防您拥有少量服务器(通常为两个)并且不希望添加更多资源。Galera仲裁...

1162
来自专栏知识分享

7-开发板接入小五物联实现远程控制(Air202模块+单片机)

这一节为教程最终版功能演示,现在不必深究,早晚自己全部都会实现的(静静的跟着我学哈)

2173
来自专栏机器之心

教程 | Python代码优化指南:从环境设置到内存分析(一)

选自pythonfiles.wordpress 机器之心编译 参与:Panda、蒋思源 近日,Python Files 博客发布了几篇主题为「Hunting P...

2869

扫码关注云+社区