saltstack returners 结果转存
returners 是saltstack对minion执行操作后，对返回的数据进行存储，可以存储到一个文件或者数据库当中。
支持的returners http://docs.saltstack.cn/zh_CN/latest/ref/returners/all/index.html 官方例子：
https://github.com/saltstack/salt/tree/develop/salt/returners 参考：http://lixcto.blog.51cto.com/4834175/1430638 http://blog.cunss.com/?p=282
我们先来试下返回到本地(返回给master屏幕)
流程：- 1.把自定义的returner传到minion端
- 2.master上执行操作，指定retuner的具体对象
- 3.minion执行命令，命令执行结果为ret表示，
- 4.这个结果一方面会按照通常的方式返回给master并在master的屏幕上面输出打印。
- 5.另一方面，ret这个结果会传给我们自定义的returner，然后在minion端执行returner，
- 6.执行returner产生的结果就是把最终的结果导入文件或者数据库 returner到minion控制台
[root@localhost ~]# cd /etc/salt/
[root@localhost salt]# mkdir _returners
[root@localhost salt]# cd _returners/
[root@localhost _returners]# cat local.py
在控制台输入执行的结果
from __future__ import absolute_import, print_function
def returner(ret):
'''
Print the return data to the terminal to verify functionality
'''
print(ret)
[root@localhost _returners]#
[root@localhost _returners]# salt '*' saltutil.sync_returners 状returner同步到minion端
[root@localhost salt]# salt-minion -l debug 在minion上启动debug模式
[root@localhost _returners]# salt '*' test.ping --return local 执行一条命令return到local的控制台
client1.roddypy.com:
True
client1.roddypy.com:
True
[root@localhost _returners]#
在minion的debug控制台中就有结果了
{'fun_args': [], 'jid': '20180306170253692413', 'return': True, 'retcode': 0, 'success': True, 'fun': 'test.ping', 'id': 'node76'}returner到minion本地文件中
[root@localhost _returners]# vim local_file.py 自定义和脚本，其实就是把ret结果保留
import json
import time
def returner(ret):
now = time.localtime()
now = time.strftime("%Y-%m-%d %H:%M:%S",now)
result_file = '/tmp/returners_roddpy.log'
result = file(result_file,'a+')
result.write('At'+str(json.dumps(now))+'\n')
result.write(str(json.dumps(ret))+'\n')
result.close()
[root@localhost _returners]#
[root@localhost _returners]# salt '*' saltutil.sync_returners 分发到minion中
client1.roddypy.com:
- returners.local_file
[root@localhost _returners]#
[root@localhost _returners]# salt '*' cmd.run 'df -h ' --return local_file
client1.roddypy.com:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 18G 1.4G 16G 9% /
tmpfs 495M 12K 495M 1% /dev/shm
/dev/sda1 485M 34M 426M 8% /boot
[root@localhost _returners]#- 查看log
[root@localhost tmp]# cat returners_roddpy.log 嘿嘿结果已经保存到minion中的/tmp/目录下了
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'node76', 'tcp://192.168.104.76:4506')
[INFO ] User root Executing command cmd.run with jid 20180306184341492564
[DEBUG ] Command details {'tgt_type': 'glob', 'jid': '20180306184341492564', 'tgt': '*', 'ret': '', 'user': 'root', 'arg': ['uptime'], 'fun': 'cmd.run'}
[INFO ] Starting a new job with PID 23218
[DEBUG ] LazyLoaded cmd.run
[INFO ] Executing command 'uptime' in directory '/root'
[DEBUG ] output: 18:43:41 up 162 days, 7:48, 4 users, load average: 0.15, 0.10, 0.12
[DEBUG ] Minion return retry timer set to 8 seconds (randomized)
[INFO ] Returning information for job: 20180306184341492564
[DEBUG ] Re-using SAuth for ('/etc/salt/pki/minion', 'node76', 'tcp://192.168.104.76:4506')returner到redis
- 1.安装redis server
[root@localhost _returners]# yum install redis -y
[root@localhost _returners]# vim /etc/redis.conf
# If you want you can bind a single interface, if the bind option is not
bind 0.0.0.0
[root@localhost _returners]#
[root@localhost _returners]# /etc/init.d/redis restart
停止 redis-server： [确定]
启动 ： [确定]
[root@localhost _returners]#
检查启动端口：
[root@localhost _returners]# netstat -nlpt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 12166/mysqld
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 14119/redis-server #已经启动
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1425/sshd
tcp 0 0 192.168.140.129:4505 0.0.0.0:* LISTEN 10378/python2.6
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1504/master
tcp 0 0 192.168.140.129:4506 0.0.0.0:* LISTEN 10400/python2.6
tcp 0 0 :::22 :::* LISTEN 1425/sshd
tcp 0 0 ::1:25 - 2.minion安装redis client
[root@localhost _returners]# salt '*' cmd.run_all 'yum install python-pip -y'
[root@localhost _returners]# salt '*' cmd.run_all 'pip install redis'
在minion上修改配置文件指定redis信息
[root@localhost tmp]# vim /etc/salt/minion
#return: mysql
redis.db: '0'
redis.host: 192.168.104.76
redis.port: 6379
重新启动minion
[root@localhost tmp]# /etc/init.d/salt-minion restart
Stopping salt-minion daemon: [确定]
Starting salt-minion daemon: [确定]
[root@localhost tmp]#
在master上执行一条命令
[root@localhost _returners]# salt '*' cmd.run 'hostname' --return redis
client1.roddypy.com:
localhost.localdomain
[root@localhost _returners]# 在master上另起一个终端看看 redis的实时信息
[root@localhost ~]# redis-cli monitor
OK
1449284517.054169 "monitor"
1449284519.768636 "SELECT" "0"
1449284519.768868 "SET" "client1.roddypy.com:20181205110159739907" "{\"fun_args\": [\"hostname\"], \"jid\": \"20181205110159739907\", \"return\": \"localhost.localdomain\", \"retcode\": 0, \"success\": true, \"fun\": \"cmd.run\", \"id\": \"client1.roddypy.com\"}"
1449284519.769302 "LPUSH" "client1.roddypy.com:cmd.run" "20181205110159739907"
1449284519.769703 "SADD" "minions" "client1.roddypy.com"
1449284519.769963 "SADD" "jids" "20181205110159739907"在master中查看redis keys
[root@localhost _returners]# redis-cli
redis 127.0.0.1:6379> ?
redis-cli 2.4.10
Type: "help @<group>" to get a list of commands in <group>
"help <command>" for help on <command>
"help <tab>" to get a list of possible help topics
"quit" to exit
redis 127.0.0.1:6379> get ?
(nil)
redis 127.0.0.1:6379> get "client1.roddypy.com:20181205110159739907"
"{\"fun_args\": [\"hostname\"], \"jid\": \"20181205110159739907\", \"return\": \"localhost.localdomain\", \"retcode\": 0, \"success\": true, \"fun\": \"cmd.run\", \"id\": \"client1.roddypy.com\"}"
redis 127.0.0.1:6379> keys * 查询所有的key
1) "client1.roddypy.com:20181205110159739907"
2) "client1.roddypy.com:20181205110142750466"
3) "minions"
4) "client1.roddypy.com:20181205110419271228"
5) "client1.roddypy.com:cmd.run"
6) "name"
7) "jids"不错，，成功进入redis db里面
returner到mysql
[root@localhost salt]# yum install mysql mysql-server -y 安装mysql
已加载插件：fastestmirror, security
设置安装进程
Loading mirror speeds from cached hostfile
* base: centos.ustc.edu.cn
* epel: ftp.cuhk.edu.hk
* extras: mirrors.sina.cn
* updates: mirrors.sina.cn
* webtatic: sp.repo.webtatic.com
包 mysql-5.6.73-5.el7_6.x86_64 已安装并且是最新版本
包 mysql-server-5.6.73-5.el7_6.x86_64 已安装并且是最新版本
无须任何处理
[root@localhost salt]# /etc/init.d/mysqld start
正在启动 mysqld： [确定]
[root@localhost salt]# /etc/init.d/mysqld status
mysqld (pid 12166) 正在运行...
[root@localhost salt]#
[root@localhost _returners]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.6.73 Source distribution
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use salt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql>CREATE DATABASE `salt`
DEFAULT CHARACTER SET utf8
DEFAULT COLLATE utf8_general_ci;
USE `salt`;
--
-- Table structure for table `jids`
--
DROP TABLE IF EXISTS `jids`;
CREATE TABLE `jids` (
`jid` varchar(255) NOT NULL,
`load` mediumtext NOT NULL,
UNIQUE KEY `jid` (`jid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
--
-- Table structure for table `salt_returns`
--
DROP TABLE IF EXISTS `salt_returns`;
CREATE TABLE `salt_returns` (
`fun` varchar(50) NOT NULL,
`jid` varchar(255) NOT NULL,
`return` mediumtext NOT NULL,
`id` varchar(255) NOT NULL,
`success` varchar(10) NOT NULL,
`full_ret` mediumtext NOT NULL,
KEY `id` (`id`),
KEY `jid` (`jid`),
KEY `fun` (`fun`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;mysql> grant all on salt.* to 'salt'@'%' identified by 'salt';
Query OK, 0 rows affected (0.03 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql>参考官方github:https://github.com/saltstack/salt/blob/develop/salt/returners/mysql.py
[root@localhost _returners]# vim mysql.py 编辑returners mysql程序
from contextlib import contextmanager
import sys
import json
import logging
try:
import MySQLdb
HAS_MYSQL = True
except ImportError:
HAS_MYSQL = False
log = logging.getLogger(__name__)
def __virtual__():
if not HAS_MYSQL:
return False
return 'mysql'
def _get_options():
'''
Returns options used for the MySQL connection.
'''
defaults = {'host': '192.168.140.129',
'user': 'salt',
'pass': 'salt',
'db': 'salt',
'port': 3306}
_options = {}
for attr in defaults:
_attr = __salt__['config.option']('mysql.{0}'.format(attr))
if not _attr:
log.debug('Using default for MySQL {0}'.format(attr))
_options[attr] = defaults[attr]
continue
_options[attr] = _attr
return _options
@contextmanager
def _get_serv(commit=False):
'''
Return a mysql cursor
'''
_options = _get_options()
conn = MySQLdb.connect(host=_options['host'], user=_options['user'], passwd=_options['pass'], db=_options['db'], port=_options['port'])
cursor = conn.cursor()
try:
yield cursor
except MySQLdb.DatabaseError as err:
error, = err.args
sys.stderr.write(error.message)
cursor.execute("ROLLBACK")
raise err
else:
if commit:
cursor.execute("COMMIT")
else:
cursor.execute("ROLLBACK")
finally:
conn.close()
def returner(ret):
'''
Return data to a mysql server
'''
with _get_serv(commit=True) as cur:
sql = '''INSERT INTO `salt_returns`
(`fun`, `jid`, `return`, `id`, `success`, `full_ret` )
VALUES (%s, %s, %s, %s, %s, %s)'''
cur.execute(sql, (ret['fun'], ret['jid'],
str(ret['return']), ret['id'],
ret['success'], json.dumps(ret)))
[root@localhost _returners]#- 操作
[root@localhost _returners]# salt '*' saltutil.sync_returners 同步到minion
[root@localhost _returners]# salt '*' cmd.run_all 'yum install MySQL-python* -y' 客户端需要安装Mysqldb模块
[root@localhost _returners]# salt '*' cmd.run 'uptime' --return mysql 执行一条命令
client1.roddypy.com:
19:20:26 up 1 day, 1:35, 2 users, load average: 0.07, 0.02, 0.00- 检查return过来的数据
[root@localhost _returners]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.6.73 Source distribution
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use salt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids |
| salt_returns |
+----------------+
2 rows in set (0.00 sec)
mysql> select * from salt_returns;
+---------+----------------------+----------------------------------------------------------------------+---------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| fun | jid | return | id | success | full_ret |
+---------+----------------------+----------------------------------------------------------------------+---------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| cmd.run | 20151206045940088527 | 19:20:26 up 1 day, 1:35, 2 users, load average: 0.07, 0.02, 0.00 | client1.roddypy.com | 1 | {"fun_args": ["uptime"], "jid": "20151206045940088527", "return": " 19:20:26 up 1 day, 1:35, 2 users, load average: 0.07, 0.02, 0.00", "retcode": 0, "success": true, "fun": "cmd.run", "id": "client1.roddypy.com"} |
+---------+----------------------+----------------------------------------------------------------------+---------------------+---------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>在高并发业务情况下，如果Minion非常多，minion同时写到数据的话，会造成mysql的压力
本文参与 腾讯云自媒体分享计划，分享自作者个人站点/博客。
原始发表：2018-03-12 ，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
目录
相关产品与服务
对象存储
对象存储（Cloud Object Storage，COS）是由腾讯云推出的无目录层次结构、无数据格式限制，可容纳海量数据且支持 HTTP/HTTPS 协议访问的分布式存储服务。腾讯云 COS 的存储桶空间无容量上限，无需分区管理，适用于 CDN 数据分发、数据万象处理或大数据计算与分析的数据湖等多种场景。