ganglia监控搭建

环境： CentOS release 6.2 (Final) 首先要下载ganglia http://sourceforge.net/projects/ganglia/files/ganglia%20monitoring%20core/3.6.0/ganglia-3.6.0.tar.gz/download?use_mirror=nchc 目前最新的为ganglia-3.7.2.tar.gz 同时要下载ganglia-web-3.71.tar.gz 下载httd http://archive.apache.org/dist/httpd/httpd-2.2.23.tar.gz php-5.4.10.tar.gz http://cn2.php.net/distributions/php-5.4.10.tar.gz 目前最新版本为：php-5.6.13.tar.gz confuse-2.7 http://www.nongnu.org/confuse/ 安装路径规划：软件名称 路径 ganglia安装路径 /usr/local/ganglia php安装路径 /usr/local/php apache安装路径 /usr/local/apache2 ganglia-web安装路径 /opt/monitor/ganglia/ganglia-web-3.5.12 rrds数据路径 /opt/monitor/ganglia/rrds 保证yum源可用 yum -y install apr-devel apr-util check-devel cairo-devel pango-devel libxml2-devel rpm-build glib2-devel dbus-devel freetype-devel fontconfig-devel gcc-c++ expat-devel python-devel libXrender-devel pkgconfig 查询是否有包未安装 yum install gcc glibc glibc-common rrdtool rrdtool-devel apr apr-devel expat expat-devel pcre pcre-devel dejavu-lgc-sans-mono-fonts dejavu-sans-mono-fonts 并且安装 pcre：安装目录/usr/local/pcre tar -xvf pcre-8.33.tar.gz ./configure --prefix=/usr/local/pcre --libdir=/usr/lib64 make make install confuse： tar -xvf confuse-2.7.tar.gz ./configure CFLAGS=-fPIC --disable-nls make make install rrdtool：安装目录/usr/local/rrdtool tar -xvf rrdtool-1.4.7.tar.gz ./configure --prefix=/usr/local/rrdtool make make install apache：安装目录/usr/local/apache2 tar -xvf httpd-2.2.23.tar.gz cd srclib cd apr ./configure --prefix=/usr/local/apr make make install cd ../apr-util ./configure --prefix=/usr/local/apr-util --with-apr=/usr/local/apr make make install cd ../../ ./configure --prefix=/usr/local/apache2 --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr-util make make install php安装 tar -xvf php-5.5.30.tar.gz ./configure --prefix=/usr/local/php --with-apxs2=/usr/local/apache2/bin/apxs 编译PHP5.5 make 时出现错误 make: *** [ext/fileinfo/libmagic/apprentice.lo] Error 1 解决办法 这是由于内存小于1G所导致. 在./configure加上选项: --disable-fileinfo Disable fileinfo support 禁用 fileinfo make make test make install php配置 从源码包目录拷贝一份php.ini-development 到php的安装目录 cp php.ini-development /usr/local/php/lib/php.ini 配置apache cd /usr/local/apache2/conf vim httpd.conf 查找字符串AddType，并增加 AddType application/x-httpd-php .php .php3 .htm .phtml .php4 查找字符串DirectoryIndex，并修改为 DirectoryIndex index.html index.php 在apache根文件中（/usr/local/apache2/htdocs/）增加一个test.php文件，<?php phpinfo(); ?>,然后访问127.0.0.1/test.php,查看结果。 重启apache,访问，

python安装：tar -xvf Python-2.7.10.tar.xz ./configure --prefix=/usr/local/python --enable-shared make make install 下面开始进行ganglia的安装(所有节点都要安装) tar -xvf ganglia-3.6.0.tar.gz ./configure --prefix=/usr/local/ganglia --with-librrd=/usr/local/rrdtool --with-python=/usr/local/python --with-libpcre=/usr/local/pcre --with-gmetad --enable-gexec --enable-status --sysconfdir=/etc/ganglia make make install 安装ganglia-web：仅主节点要装 解压，配置 cd ganglia-web-3.5.12/ cp conf_default.php conf.php 修改：$conf['rrdtool'] = "/usr/bin/rrdtool";-->$conf['rrdtool'] = "/usr/local/rrdtool/bin/rrdtool"; 修改：$conf['gweb_confdir'] = "/var/lib/ganglia-web";-->$conf['gweb_confdir'] = "/opt/monitor/ganglia/ganglia-web-3.5.12"; 修改：$conf['gmetad_root'] = "/var/lib/ganglia";-->$conf['gmetad_root'] = "/opt/monitor/ganglia"; 修改header.php <?php session_start(); ini_set('date.timezone','PRC'); --修改时区为本地时区(新加这一行) if (isset($_GET['date_only'])) { $d = date("r"); echo $d; exit(0); } 配置临时目录 cd /opt/monitor/ganglia/ganglia-web-3.5.12/dwoo/ mkdir cache chmod 777 cache mkdir compiled chmod 777 compiled 使用apache发布ganglia-web（主节点） vi /usr/local/apache2/conf/httpd.conf 启动httpd服务 ? 1 /usr/local/apache2/bin/apachectl restart 配置ganglia 配置gmetad(主节点配置) cd /opt/monitor/ganglia-3.6.0 cp gmetad/gmetad.init /etc/init.d/gmetad 注：这里编译ganglia的时候加了参数sysconfdir，所以这里不需要进行如下步骤 cp gmetad/gmetad.conf /usr/local/ganglia/etc/ 但是如果没有加上系统配置参数，则要执行上一步，然后 gmetad -t | tee /usr/loca/ganglia/etc/gmetad.conf这个是为了生成默认配置文件 vim /etc/init.d/gmetad 修改：GMETAD=/usr/sbin/gmetad-->GMETAD=/usr/local/ganglia/sbin/gmetad vim etc/ganglia/gmetad.conf 修改：data_source "my cluster" localhost --> data_source "solrCloud" localhost 192.168.36.82:8649 192.168.36.81:8649 192.168.36.84:8649 192.168.36.80:8649 放开xml_port这里的端口不能和data_source的端口冲突 放开interactive_port vim opt/monitor/ganglia/ganglia-web-3.5.12/conf.php

这个端口和interactive port端口要一致 修改rrd_rootdir的路径 # rrd_rootdir "/some/other/place"-->rrd_rootdir "/opt/monitor/ganglia/rrds" 查看case_sensitive_hostnames 0是否已经放开，并且值为0 修改# gridname "MyGrid"-->gridname "solrCloud" 修改rrds数据目录所有者 chown -R nobody:nobody /etc/monitor/ganglia/rrds 启动gmetad服务，并设为开机自动运行 service gmetad restart Starting GANGLIA gmetad: [ OK ] chkconfig --add gmetad gmond配置（所有节点） cd /opt/monitor/ganglia-3.6.0 cp gmond/gmond.init /etc/init.d/gmond vim /etc/init.d/gmond 修噶:#GMOND=/usr/sbin/gmond-->GMOND=/usr/local/ganglia/sbin/gmond ./gmond/gmond -t > /etc/ganglia/gmond.conf（生成默认的配置文件） 或者gmond -t | tee /etc/ganglia/gmond.conf # generate initial gmond config（生成默认的配置文件） vim /etc/ganglia/gmond.conf 参考如下进行修改 gmetad 注：这个数据源后面的ip的是配置的server的ip data_source "Namenode" 192.168.1.28:8653 data_source "Datanode" 192.168.1.27:8649 data_source "Portal" 192.168.1.43:8650 data_source "Collector" 192.168.1.35:8651 data_source "DB" 192.168.1.51:8652 gridname "Hadoop" rrd_rootdir "/opt/modules/ganglia/html/rrds" #配置rrd数据保存文件的路径，给web界面用的，这个是固定的，最好放在web文件夹下，并赋予正确的权限 case_sensitive_hostnames 0 数据来源有5个，这5个分别是每个组的组长，相当于一道杠。但是组长是不需要配置gmetad的，除非你要做多级组播收集数据。每个组长只需要分配不同的端口号就可以了。你可能会问，IP不一样，端口一样不行吗？不行，因为这个IP是单播IP，相当于一个路由指向，而Ganglia实际的数据传输是在多播IP上进行的，而多播IP只有一个。在客户端配置，如果你需要多级gmetad，可以配多个多播IP。 客户端配置就比较复杂一些了。我只贴上需要修改的部分，其他都是默认就可以了 注：这里是多播的模式配置 gmond cluster { name = "Portal" #对应gmetad中的Portal，名称一定要写对。 owner = "unspecified"---->nobody latlong = "unspecified" url = "unspecified" } /* Feel free to specify as many udp_send_channels as you like. Gmond used to only support having a single channel */ udp_send_channel { #bind_hostname = yes # Highly recommended, soon to be default. # This option tells gmond to use a source address # that resolves to the machine's hostname. Without # this, the metrics may appear to come from any # interface and the DNS names associated with # those IPs will be used to create the RRDs. mcast_join = 239.2.11.71 port = 8650 #gmetad中的Portal所分配的端口号。 ttl = 1 } /* You can specify as many udp_recv_channels as you like as well. */ udp_recv_channel { mcast_join = 239.2.11.71 port = 8650 bind = 239.2.11.71 } /* You can specify as many tcp_accept_channels as you like to share an xml description of the state of the cluster */ tcp_accept_channel { port = 8650 } 红色部分就是Portal小组的端口，从gmetad.conf中可以看到，Portal小组属于8650端口，那么相应的在gmond中，也要将udp和tcp端口写为8650。 如果是另外一个组的，就写上在gmetad中配置的那个端口。当然，你可以把这个端口号想像为小组的代号。可能更好理解一些。 再加上另外一个组的成员gmond就更容易理解了 cluster { name = "DB" owner = "unspecified" latlong = "unspecified" url = "unspecified" } /* The host section describes attributes of the host, like the location */ host { location = "unspecified" } /* Feel free to specify as many udp_send_channels as you like. Gmond used to only support having a single channel */ udp_send_channel { #bind_hostname = yes # Highly recommended, soon to be default. # This option tells gmond to use a source address # that resolves to the machine's hostname. Without # this, the metrics may appear to come from any # interface and the DNS names associated with # those IPs will be used to create the RRDs. mcast_join = 239.2.11.71 port = 8652 ttl = 1 } /* You can specify as many udp_recv_channels as you like as well. */ udp_recv_channel { mcast_join = 239.2.11.71 port = 8652 bind = 239.2.11.71 } /* You can specify as many tcp_accept_channels as you like to share an xml description of the state of the cluster */ tcp_accept_channel { port = 8652 } 红色对红色，蓝色对蓝色。一目了然。 复制python module到ganglia部署目录 mkdir /usr/local/ganglia/lib64/ganglia/python_modules cp ./gmond/python_modules/*/*.py /usr/local/ganglia/lib64/ganglia/python_modules 安装程序ganglia-3.6.0默认提供了一些python module的配置文件,只需要部署到 /usr/local/ganglia/etc/conf.d目录下面即可生效 cp ./gmond/python_modules/conf.d/*.pyconf /etc/ganglia/conf.d service gmond restart chkconfig --add gmond 下面贴上单播的模式配置： 单播模式设置： 在每个节点上都要配置/etc/ganglia/gmond.conf globals { daemonize = yes setuid = yes user = root /*运行Ganglia的用户(一般可能是nobody)*/ debug_level = 0 max_udp_msg_len = 1472 mute = no deaf = no host_dmax = 120 /*secs */ cleanup_threshold = 300 /*secs */ gexec = no send_metadata_interval = 15 /*发送数据的时间间隔*/ } cluster { name = "hadoop_hf" /*集群名称*/ owner = "root" /*运行Ganglia的用户*/ latlong = "unspecified" url = "unspecified" } udp_send_channel { # mcast_join = 239.2.11.71 /*注释掉组播*/ host = 192.168.228.172 /*发送给安装gmetad的机器*/ port = 8649 ttl = 1 } /* You can specify as many udp_recv_channels as you like as well. */ udp_recv_channel { # mcast_join = 239.2.11.71 /*注释掉组播*/ port = 8649 # bind = 239.2.11.71 /*注释掉组播*/ } 测试： # 命令行打印当前活动client gstat -a 常见错误： err2: There was an error collecting ganglia data (127.0.0.1:8652): fsockopen error: Connection refused 解决代码： # mkdir -p /var/lib/ganglia/rrds # chown nobody:nobody /var/lib/ganglia/rrds 注：注：当有信息出错时，可以查看log文件 tail –n 100 /var/log/message //查看对进的100行 echo /dev/null > /var/log/message //清空message 参考文献： http://my.oschina.net/duangr/blog/181585 http://my.oschina.net/duangr/blog/181189 http://blog.csdn.net/kafeiflynn/article/details/6634380 http://www.cnblogs.com/Richardzhu/p/3481996.html