https://github.com/AliyunContainerService/kube-eventer
kube-event是一个事件发射器,将kubernetes事件发送到sink (或者dingtalk,sls,kafka 企业微信等)。
kubernetes的核心设计理念是状态机。因此,当转换到所需状态时将出现Normal事件,而当转换到意外状态时将出现Warning事件。kube-event可以帮助诊断、分析和报警问题。
我下面是是私人的dingtalk机器人的token获取方法如下图:
cat aliyun-kube-eventer.deploy.yaml 内容如下:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: kube-eventer
name: kube-eventer
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: kube-eventer
template:
metadata:
labels:
app: kube-eventer
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
dnsPolicy: ClusterFirstWithHostNet
serviceAccount: kube-eventer
containers:
- image: registry.aliyuncs.com/acs/kube-eventer-amd64:v1.2.0-484d9cd-aliyun
name: kube-eventer
command:
- "/kube-eventer"
- "--source=kubernetes:https://kubernetes.default"
## .e.g,dingtalk sink demo
- --sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=222222222222222222222222222222222222222222&label=test-k8s&level=Warning&msg_type=markdown
env:
# If TZ is assigned, set the TZ value as the time zone
- name: TZ
value: "Asia/Shanghai"
volumeMounts:
- name: localtime
mountPath: /etc/localtime
readOnly: true
- name: zoneinfo
mountPath: /usr/share/zoneinfo
readOnly: true
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 500m
memory: 250Mi
volumes:
- name: localtime
hostPath:
path: /etc/localtime
- name: zoneinfo
hostPath:
path: /usr/share/zoneinfo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kube-eventer
rules:
- apiGroups:
- ""
resources:
- configmaps
- events
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kube-eventer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kube-eventer
subjects:
- kind: ServiceAccount
name: kube-eventer
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-eventer
namespace: kube-system
> kubectl apply -f aliyun-kube-eventer.deploy.yaml
> kubectl get pods -n kube-system| grep kube-event
kube-eventer-5dbcb78f59-wsjld 1/1 Running 0 11m
> kubectl logs -n kube-system kube-eventer-5dbcb78f59-wsjld
I0627 15:03:40.668622 1 eventer.go:67] /kube-eventer --source=kubernetes:https://kubernetes.default --sink=dingtalk:https://oapi.dingtalk.com/robot/send?access_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&label=dbatest-k8s&level=Warning&msg_type=markdown
I0627 15:03:40.668693 1 eventer.go:68] kube-eventer version: v1.2.0 commit: 484d9cd
I0627 15:03:40.670438 1 eventer.go:94] Starting with DingTalkSink sink
I0627 15:03:40.670474 1 eventer.go:108] Starting eventer
I0627 15:03:40.670715 1 eventer.go:116] Starting eventer http service
I0627 15:04:00.001057 1 manager.go:102] Exporting 3 events
I0627 15:04:30.000654 1 manager.go:102] Exporting 11 events
I0627 15:05:00.000131 1 manager.go:102] Exporting 12 events
I0627 15:05:30.000129 1 manager.go:102] Exporting 0 events
I0627 15:06:00.000130 1 manager.go:102] Exporting 0 events
I0627 15:06:30.000121 1 manager.go:102] Exporting 0 event
然后,我们执行一个暴力删除的操作(delete pod -n default web-0 --force --grace-period 0), 模拟一个异常事件出来。
稍等片刻,钉钉就会收到提示如下:
当然,我们的event消息也可以配置让它落到kafka中,后续通过自定义的程序去消费做各种各样的操作,也便于通过logstash消费kafka将数据入到ES中,便于做统计汇总。
如果要接到企业微信,可以使用这样配置:
--sink=wechat:?corp_id=wwxxxxxxxxxx1a&corp_secret=xxxxxxxxxxxxxxxxxxxxxxxxxxx4U&agent_id=10xxxxxx7&to_user=&level=Warning&label=K8S-Alert-Prod&msg_type=markdown
参考:
https://github.com/AliyunContainerService/kube-eventer
https://www.imooc.com/article/301284