k8s对接云实现自动扩容pod
介绍
在之前的文章我介绍了下 Custom Metric 怎么实现自动扩容的。k8s基于自定义指标实现自动扩容
实际上Kubernetes定义了三种不同的监控数据接口，分别是Resource Metric，Custom Metric以及External Metric。
一般来说Resource Metric是通过metrics-server采集；
Custom Metric是通过prometheus来实现自定义扩容。
External Metric就是针对云场景的了，比方说通过获取slb最大连接数来实现自动扩容。
下面我来说下具体怎么实现的。
部署
安装alibaba-cloud-metrics-adapter，以下是yaml文件
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: alibaba-cloud-metrics-adapter
name: alibaba-cloud-metrics-adapter
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: alibaba-cloud-metrics-adapter
template:
metadata:
labels:
app: alibaba-cloud-metrics-adapter
name: alibaba-cloud-metrics-adapter
spec:
serviceAccountName: admin
containers:
- name: alibaba-cloud-metrics-adapter
image: registry.cn-beijing.aliyuncs.com/acs/alibaba-cloud-metrics-adapter-amd64:v0.2.0-alpha-e8f8c17f
imagePullPolicy: IfNotPresent
ports:
- containerPort: 443
name: https
- containerPort: 8080
name: http
volumeMounts:
- mountPath: /tmp
name: temp-vol
- name: tz-config
mountPath: /etc/localtime
readOnly: true
volumes:
- name: temp-vol
emptyDir: {}
- name: tz-config
hostPath:
path: /etc/localtime
---
apiVersion: v1
kind: Service
metadata:
name: alibaba-cloud-metrics-adapter
namespace: kube-system
spec:
ports:
- name: https
port: 443
targetPort: 443
- name: http
port: 80
targetPort: 8080
selector:
app: alibaba-cloud-metrics-adapter
---
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
name: v1beta1.external.metrics.k8s.io
spec:
service:
name: alibaba-cloud-metrics-adapter
namespace: kube-system
group: external.metrics.k8s.io
version: v1beta1
insecureSkipTLSVerify: true
groupPriorityMinimum: 100
versionPriority: 100
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: external-metrics-server-resources
rules:
- apiGroups:
- external.metrics.k8s.io
resources: ["*"]
verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: external-metrics-resource-reader
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-metrics-server-resources
subjects:
- kind: ServiceAccount
name: horizontal-pod-autoscaler
namespace: kube-system可以使用下面的命令来检测是否生效了：
# kubectl get --raw="/apis/external.metrics.k8s.io/v1beta1" | jq
{
"kind": "APIResourceList",
"apiVersion": "v1",
"groupVersion": "external.metrics.k8s.io/v1beta1",
"resources": [
{
"name": "sls_ingress_qps",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "slb_l4_connection_utilization",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "slb_l7_qps",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "ahas_sentinel_total_qps",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "ahas_sentinel_avg_rt",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "k8s_workload_cpu_util",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "k8s_workload_memory_request",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
{
"name": "k8s_workload_memory_cache",
"singularName": "",
"namespaced": true,
"kind": "ExternalMetricValueList",
"verbs": [
"get"
]
},
。。。。。。。。。简单说下各个指标的含义，方便之后去选择哪个指标去自动扩容。
❝slb_l4_traffic_rx 每秒流入 slb_l4_packet_tx 每秒流入的数据包数 slb_l4_active_connection 活动连接 slb_l4_max_connection 最大连接数 slb_l7_qps QPS slb_l7_status_2xx 2xx个请求（每秒） slb_l7_upstream_4xx 上游服务4xx请求（每秒） sls_ingress_qps 特定入口路由的QPS sls_ingress_inflow 入口流入带宽 k8s_workload_memory_usage 内存使用情况 k8s_workload_memory_rss rss ❞
扩容例子
根据slb_l4_active_connection这个指标，实现自动扩容
apiVersion: apps/v1beta2 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment-basic
labels:
app: nginx
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9 # replace it with your exactly <image_name:tags>
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
spec:
externalTrafficPolicy: Local
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
sessionAffinity: None
type: LoadBalancer
---
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: slb-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1beta2
kind: Deployment
name: nginx-deployment-basic
minReplicas: 5
maxReplicas: 10
metrics:
- type: External
external:
metric:
name: slb_l4_active_connection
selector:
matchLabels:
# slb.instance.id: "lb-2ze2locy5fk8at1cfx47y"
slb.instance.id: ""
# slb.instance.port: "80"
slb.instance.port: ""
target:
type: Value
value: 100这样就实现了通过External Metric自动扩容。
参考链接
https://github.com/AliyunContainerService/alibaba-cloud-metrics-adapter
本文参与 腾讯云自媒体分享计划，分享自微信公众号。
原始发表：2020-06-28，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读