prometheus+grafana 对于现在这个时间点来说,相信很多同行都应该已经开始玩起来了,当仍然可能有一部分人可能还不知道prometheus+grafana 的存在,也可能还有一部分人虽然知道它的存在,但却懒于动手,如果是这样,那后面的内容你可得打起精神来了,因为可能你会被grafana炫酷的视觉效果惊艳到。
让我们一起了解一下吧。
首先,简单介绍下prometheus+grafana 这对鸳鸯到底是什么:
prometheus 是由 SoundCloud 开发的开源监控报警系统和时序列数据库(TSDB),prometheus是一个监控采集与数据存储框架(监控server端),具体采集什么数据依赖于具体的exporter(监控client端),例如:采集MySQL的数据需要使用到mysql_exporter,prometheus调用mysql_expoter采集到mysql的监控指标之后,把mysql_exporter采集到的数据存放到prometheus所在服务器的磁盘数据文件中保存。它的各个组件基本都是用 golang 编写,对编译和部署十分友好.并且没有特殊依赖,基本都是独立工作。以下是prometheus架构图(图片来源:https://prometheus.io/docs/introduction/overview/)
grafana是一个高颜值的监控绘图程序,也是一个可视化面板(Dashboard),grafana的厉害之处除了高颜值,还支持多种数据源(支持Graphite、zabbix、InfluxDB、Prometheus和OpenTSDB作为数据源)、支持灵活丰富的dashboard配置选项(例如:可以把多个实例的相同采集项配置在一个展示框里),使得相较于其他开源监控系统来说更易用性,学习成本更低。从视觉上来说,比以往的任何开源的监控系统都看起来要养眼很多,下面先看两张监控效果图:
[root@localhost ~]# mkdir /data[root@localhost ~]# tar xvf prometheus-2.1.0.linux-amd64.tar.gz -C /data/
[root@localhost ~]# tar xf node_exporter-0.15.2.linux-amd64.tar -C /root/# 如果需要监控mysql,则继续解压mysql_exporter[root@localhost ~]# tar xf mysqld_exporter-0.10.0.linux-amd64.tar -C /root/
[root@localhost ~]# cd /data/[root@localhost data]# mv prometheus-2.1.0.linux-amd64/ prometheus[root@localhost ~]# cd /data/prometheus
[root@localhost data]# cat prometheus.yml # my global configglobal: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s).# A scrape configuration containing exactly one endpoint to scrape:# Here it's Prometheus itself.scrape_configs:- file_sd_configs: - files: - host.yml job_name: Host metrics_path: /metrics relabel_configs: - source_labels: [__address__] regex: (.*) target_label: instance replacement: $1 - source_labels: [__address__] regex: (.*) target_label: __address__ replacement: $1:9100- file_sd_configs: files: - mysql.yml job_name: MySQL metrics_path: /metrics relabel_configs: - source_labels: [__address__] regex: (.*) target_label: instance replacement: $1 - source_labels: [__address__] regex: (.*) target_label: __address__ replacement: $1:9104
- job_name: prometheus static_configs: - targets: - localhost:9090
[root@localhost prometheus]# /data/prometheus/prometheus --storage.tsdb.retention=30d &
# 修改WorkingDirectory参数为你的prometheus的工作目录[root@localhost prometheus]# cat /usr/lib/systemd/system/prometheus.service[Unit]Description=Prometheus instanceWants=network-online.targetAfter=network-online.targetAfter=postgresql.service mariadb.service mysql.service
[Service]User=rootGroup=rootType=simpleRestart=on-failureWorkingDirectory=/data/prometheus/RuntimeDirectory=prometheusRuntimeDirectoryMode=0750ExecStart=/data/prometheus/prometheus --storage.tsdb.retention=30d --config.file=/data/prometheus/prometheus.ymlLimitNOFILE=10000TimeoutStopSec=20
[Install]WantedBy=multi-user.target
[root@localhost ~]# tar xf grafana-4.6.3.linux-x64.tar.gz -C /data/prometheus/[root@localhost ~]# cd /data/prometheus[root@localhost prometheus]# mv grafana-4.6.3/ grafana
[root@localhost ]# cd /data/prometheus/grafana[root@localhost ]# ./bin/grafana-server
[root@localhost service]# cat /usr/lib/systemd/system/grafana-server.service[Unit]Description=Grafana instanceDocumentation=http://docs.grafana.orgWants=network-online.targetAfter=network-online.targetAfter=postgresql.service mariadb.service mysql.service[Service]User=rootGroup=rootType=simpleRestart=on-failureWorkingDirectory=/data/prometheus/grafanaRuntimeDirectory=grafanaRuntimeDirectoryMode=0750ExecStart=/data/prometheus/grafana/bin/grafana-serverLimitNOFILE=10000TimeoutStopSec=20[Install]WantedBy=multi-user.target
[root@localhost ~]# tar xvf grafana-dashboards-1.6.1.tar.gz [root@localhost ~]# cd grafana-dashboards-1.6.1[root@localhost grafana-dashboards-1.6.1]# updatedb [root@localhost grafana-dashboards-1.6.1]# locate json |grep dashboards/............/root/grafana-dashboards-1.6.1/dashboards/CPU_Utilization_Details_Cores.json/root/grafana-dashboards-1.6.1/dashboards/Disk_Performance.json/root/grafana-dashboards-1.6.1/dashboards/Disk_Space.json............/root/grafana-dashboards-1.6.1/dashboards/MySQL_InnoDB_Metrics.json/root/grafana-dashboards-1.6.1/dashboards/MySQL_InnoDB_Metrics_Advanced.json............/root/grafana-dashboards-1.6.1/dashboards/MySQL_Overview.json/root/grafana-dashboards-1.6.1/dashboards/MySQL_Performance_Schema.json............/root/grafana-dashboards-1.6.1/dashboards/MySQL_Replication.json/root/grafana-dashboards-1.6.1/dashboards/MySQL_Table_Statistics.json............/root/grafana-dashboards-1.6.1/dashboards/Summary_Dashboard.json/root/grafana-dashboards-1.6.1/dashboards/System_Overview.json............
[root@localhost ~]# tar xf node_exporter-0.15.2.linux-amd64.tar [root@localhost ~]# mv node_exporter-0.15.2.linux-amd64 node_exporter
[root@localhost ~]# cd node_exporter[root@localhost node_exporter]# nohup ./node_exporter &
[root@localhost node_exporter]# cat /data/prometheus/host.yml- labels: service: test targets: - 10.10.30.165
[root@localhost mysqld_exporter]# cat /data/prometheus/host.yml- labels: service: test targets: - 10.10.30.165 - 10.10.20.14
[root@localhost ~]# tar xf mysqld_exporter-0.10.0.linux-amd64.tar [root@localhost ~]# mv mysqld_exporter-0.10.0.linux-amd64 mysqld_exporter
[root@luoxiaobo-01 ~]# export DATA_SOURCE_NAME='admin:password@(10.10.20.14:3306)/'[root@luoxiaobo-01 ~]# echo "export DATA_SOURCE_NAME='admin:password@(10.10.20.14:3306)/'" >> /etc/profile
# 由于目前最新的版本默认关闭了大量的mysql采集项,需要显式使用相应的选项开启(截止到写稿时间,最新的开发版本可以通过prometheus端的配置项让exporter端生效,而无需再exporter中使用大量的启动选项开启)[root@localhost ~]# cd mysqld_exporter[root@localhost mysqld_exporter]# nohup ./mysqld_exporter --collect.info_schema.processlist --collect.info_schema.innodb_tablespaces --collect.info_schema.innodb_metrics --collect.perf_schema.tableiowaits --collect.perf_schema.indexiowaits --collect.perf_schema.tablelocks --collect.engine_innodb_status --collect.perf_schema.file_events --collect.info_schema.processlist --collect.binlog_size --collect.info_schema.clientstats --collect.perf_schema.eventswaits &
[root@localhost mysqld_exporter]# cat /data/prometheus/host.yml- labels: service: test targets: - 10.10.30.165 - 10.10.20.14
[root@localhost mysqld_exporter]# cat /data/prometheus/mysql.yml
- labels: service: test targets: - 10.10.30.165 - 10.10.20.14
关于「3306π」社区
围绕 MySQL 核心技术,将互联网行业中最重要的数据化解决方案带到传统行业中;囊括其他开源技术Redis、MongoDB、Hbase、Hadoop、ElasticSearch、Storm、Spark等;分享干货知识,即便是赞助商,也要求如此,拒绝放水。