Ambari集成Elasticsearch 5.x

程裕强

发布于 2018-01-02 16:53:54

2.2K1

发布于 2018-01-02 16:53:54

文章被收录于专栏：大数据学习笔记大数据学习笔记大数据学习笔记

1、资源URL

https://github.com/Jaraxal/ambari-elasticsearch-service 下载得到ambari-elasticsearch-service-master.zip压缩包

2、首先添加服务的stack

（1）进入ambari源码ambari-server/resources/stacks/HDP/2.4/services目录

[root@master services]# pwd
/var/lib/ambari-server/resources/stacks/HDP/2.4/services

[root@master services]# ls

ACCUMULO  FALCON  HBASE  HIVE   KERBEROS  MAHOUT  PIG     RANGER_KMS  SPARK  stack_advisor.py   stack_advisor.pyo  TEZ   ZOOKEEPER

ATLAS     FLUME   HDFS   KAFKA  KNOX      OOZIE   RANGER  SLIDER      SQOOP  stack_advisor.pyc  STORM              YARN

（2）创建ELASTICSEARCH目录并进入该目录

[root@master services]# mkdir ELASTICSEARCH
[root@master services]# cd ELASTICSEARCH/

（3）编辑metainfo.xml

[root@master ELASTICSEARCH]#vi metainfo.xml
[root@master ELASTICSEARCH]#cat metainfo.xml
<?xml version="1.0"?>
    <metainfo>
      <schemaVersion>2.0</schemaVersion>
      <services>
        <service>
          <name>ELASTICSEARCH</name>
          <version>5.1.1</version>
          <extends>common-services/ELASTICSEARCH/5.1.1</extends>
        </service>
      </services>
    </metainfo>

3、ambari-server/src/main/resources/common-services

（1）进入ambari-server/src/main/resources/common-services目录

[root@master common-services]# pwd
/opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services

[root@master common-services]# ls
ACCUMULO        ATLAS          FLUME    HBASE  KAFKA     LOGSEARCH  PIG     RANGER_KMS  SPARK2  TEZ       ZOOKEEPER

AMBARI_INFRA    ELASTICSEARCH  GANGLIA  HDFS   KERBEROS  MAHOUT     PXF     SLIDER      SQOOP   YARN

AMBARI_METRICS  FALCON         HAWQ     HIVE   KNOX      OOZIE      RANGER  SPARK       STORM   ZEPPELIN

（2）创建ES版本目录

[root@master common-services]# mkdir -p ELASTICSEARCH/5.1.1/

[root@vnode1 common-services]# cd

（3）解压ambari-elasticsearch-service-master.zip

[root@master ~]# unzip ambari-elasticsearch-service-master.zip

（4）编辑ambari-elasticsearch-service-master中的metainfo.xml文件

[root@master ~]# cd ambari-elasticsearch-service-master    

[root@master ambari-elasticsearch-service-master]# vi metainfo.xml
[root@vnode0 5.1.1]# vi metainfo.xml

<?xml version="1.0"?>
<metainfo>
    <schemaVersion>2.0</schemaVersion>
    <services>
        <service>
            <name>ELASTICSEARCH</name>
            <displayName>ElasticSearch</displayName>
            <comment>ElasticSearch service</comment>
            <version>5.1.1</version>
            <components>
                <component>
                    <name>ELASTICSEARCH</name>
                    <displayName>ElasticSearch</displayName>
                    <category>MASTER</category>
                    <cardinality>1</cardinality>
                    <commandScript>
                        <script>scripts/master.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                </component>
                <component>
                    <name>ELASTICSEARCH_NODE</name>
                    <displayName>ElasticSearchNode</displayName>
                    <category>SLAVE</category>
                    <cardinality>ALL</cardinality>
                    <!--自动部署，仅在cardinality满足的情况下，界面上就不问，在这台主机，是否安装这个组件了-->
                    <auto-deploy> 
                        <enabled>false</enabled>
                    </auto-deploy>
                    <commandScript>
                        <script>scripts/slave.py</script>
                        <scriptType>PYTHON</scriptType>
                        <timeout>600</timeout>
                    </commandScript>
                    <!--<configFiles>
                    <configFile>
                        <type>env</type>
                        <fileName>elasticsearch-env.xml</fileName>
                        <dictionaryName>elasticsearch-env</dictionaryName>
                     </configFile>
                    </configFiles>-->
                </component>
            </components>
            <osSpecifics>
                <osSpecific>
                  <osFamily>any</osFamily>
                  <packages>
                    <package>
                      <name>elasticsearch</name>
                    </package>
                  </packages>
                </osSpecific>
            </osSpecifics>
            <requiredServices>
                <service>GANGLIA</service> <!--依赖服务，安装的时候限制，没有不能继续安装-->
            </requiredServices>
            <configuration-dependencies>
              <config-type>elasticsearch-env</config-type>
            </configuration-dependencies>
            <monitoringService>false</monitoringService> <!--如果是监控服务的话，每个组件重启，他都要重启-->
        </service>
    </services>
</metainfo>

（5）复制脚本将https://github.com/Jaraxal/ambari-elasticsearch-service下载得到ambari-elasticsearch-service-master.zip压缩包的配置文件复制到ambari对应目录下。

[root@master ambari-elasticsearch-service-master]# cp -r * /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

（6）查看

[root@master ambari-elasticsearch-service-master]# cd /opt/apache-ambari-2.4.2-src/ambari-server/src/main/resources/common-services/ELASTICSEARCH/5.1.1/

[root@master 5.1.1]# ls

configuration  LICENSE  metainfo.xml  package  quicklinks  README.md

[root@master 5.1.1]#

（7）设置ES下载URL

[root@master 5.1.1]# cd package/scripts/

[root@master scripts]# ls

elastic_common.py  master.py  params.py  status_params.py

[root@master scripts]# vi params.py 

elastic_download = 'http://192.168.1.180/hdp/elasticsearch-5.1.1.tar.gz'

[root@master 2.4.3.0]# vi HDP-2.4.3.0-227.xml

<?xml version="1.0"?>

<repository-version xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="version_definition.xsd">

  <release>

    <type>STANDARD</type>

    <stack-id>HDP-2.4</stack-id>

    <version>2.4.3.0</version>

    <build>227</build>

    <compatible-with>2.4.[0-9]+</compatible-with>

    <release-notes>http://example.com</release-notes>

    <display>HDP-2.4.3.0-227</display>

    <package-version>2_4_3_0_*</package-version>

  </release>

  <manifest>

    <service id="KNOX-060" name="KNOX" version="0.6.0"/>

    <service id="SLIDER-0800" name="SLIDER" version="0.80.0"/>

    <service id="FALCON-061" name="FALCON" version="0.6.1"/>

    <service id="MAHOUT-090" name="MAHOUT" version="0.9.0"/>

    <service id="HBASE-112" name="HBASE" version="1.1.2"/>

    <service id="FLUME-152" name="FLUME" version="1.5.2"/>

    <service id="HIVE-121000" name="HIVE" version="1.2.1000"/>

    <service id="TEZ-070" name="TEZ" version="0.7.0"/>

    <service id="OOZIE-420" name="OOZIE" version="4.2.0"/>

    <service id="HDFS-271" name="HDFS" version="2.7.1"/>

    <service id="YARN-271" name="YARN" version="2.7.1"/>

    <service id="MAPREDUCE2-271" name="MAPREDUCE2" version="2.7.1"/>

    <service id="ZOOKEEPER-346" name="ZOOKEEPER" version="3.4.6"/>

    <service id="KAFKA-090" name="KAFKA" version="0.9.0"/>

    <service id="SQOOP-146" name="SQOOP" version="1.4.6"/>

    <service id="ATLAS-050" name="ATLAS" version="0.5.0"/>

    <service id="SPARK-162" name="SPARK" version="1.6.2"/>

    <service id="STORM-0100" name="STORM" version="0.10.0"/>

    <service id="PIG-0150" name="PIG" version="0.15.0"/>

    <service id="RANGER-050" name="RANGER" version="0.5.0"/>

    <service id="RANGER_KMS-050" name="RANGER_KMS" version="0.5.0"/>

    <service id="ACCUMULO-170" name="ACCUMULO" version="1.7.0"/>

    <service id="ELASTICSEARCH-511" name="ELASTICSEARCH" version="5.1.1"/>

  </manifest>

  <available-services/>

  <repository-info>

    <os family="redhat6">

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/2.x/BUILDS/2.4.3.0-227</baseurl>

        <repoid>HDP-2.4</repoid>

        <reponame>HDP</reponame>

      </repo>

      <repo>

        <baseurl>http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos7</baseurl>

        <repoid>HDP-UTILS-1.1.0.20</repoid>

        <reponame>HDP-UTILS</reponame>

      </repo>

    </os>

  </repository-info>

</repository-version>

[root@master 2.4.3.0]# createrepo --update .

Spawning worker 0 with 46 pkgs

Spawning worker 1 with 46 pkgs

Spawning worker 2 with 45 pkgs

Spawning worker 3 with 45 pkgs

Workers Finished

Saving Primary metadata

Saving file lists metadata

Saving other metadata

Generating sqlite DBs

Sqlite DBs complete

本文参与腾讯云自媒体分享计划，分享自作者个人站点/博客。

原始发表：2017-12-13 ，如有侵权请联系 cloudcommunity@tencent.com 删除

其他

本文分享自作者个人站点/博客前往查看

如有侵权，请联系 cloudcommunity@tencent.com 删除。

本文参与腾讯云自媒体分享计划，欢迎热爱写作的你一起参与！

其他

登录后参与评论

0 条评论

热度