Kubernetes 1.9集群使用traefik发布服务

星哥玩云

发布于 2022-07-13 11:30:51

4650

发布于 2022-07-13 11:30:51

文章被收录于专栏：开源部署

在前文中介绍了在kubernetes 1.5.2集群环境中使用traefik进行服务发布。Traefik采用daemonset方式部署，连接api-server走的是http协议，也未配置rbac。本文将介绍在k8s 1.9版本中使用deployment方式部署traefik来进行服务发布。

在开始之前，需要先了解一下什么是RBAC。RBAC（基于角色的访问控制）使用 rbac.authorization.k8s.io API 组来实现权限控制，RBAC 允许管理员通过 Kubernetes API 动态的配置权限策略。在 1.6 版本中 RBAC 还处于 Beat 阶段，如果想要开启 RBAC 授权模式需要在 apiserver 组件中指定 --authorization-mode=RBAC 选项。

在 RBAC API 的四个重要概念： Role：是一系列的权限的集合，例如一个角色可以包含读取 Pod 的权限和列出 Pod 的权限 ClusterRole：跟 Role 类似，但是可以在集群中到处使用（ Role 是 namespace 一级的） RoloBinding：把角色映射到用户，从而让这些用户继承角色在 namespace 中的权限。 ClusterRoleBinding：让用户继承 ClusterRole 在整个集群中的权限。

简单点说RBAC实现了在k8s集群中对api-server的鉴权，更多的RBAC知识点请查阅官方文档：https://kubernetes.io/docs/admin/authorization/rbac/

一、给集群的节点打上label 因为选择deployment方式部署，所以要给集群的节点打上label，后续选择nodeSelector指定traefik=proxy，副本数和集群节点数一致的时候，所有的节点上都会运行一个pod

# kubectl get nodes --show-labels # kubectl label node vm1 traefik=proxy # kubectl label node vm2 traefik=proxy # kubectl get nodes --show-labels

二、准备yaml文件 1、rbac文件

# cat traefik-rbac.yaml --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system

在启用rbac的环境下，如果鉴权未配置清楚，则traefik pod会报错如下

E0226 00:15:27.729832 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list services at the cluster scope E0226 00:15:29.013298 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:default" cannot list endpoints at the cluster scope E0226 00:15:29.213354 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:kube-system:default" cannot list secrets at the cluster scope E0226 00:15:29.698574 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1beta1.Ingress: ingresses.extensions is forbidden: User "system:serviceaccount:kube-system:default" cannot list ingresses.extensions at the cluster scope E0226 00:15:30.411837 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: services is forbidden: User "system:serviceaccount:kube-system:default" cannot list services at the cluster scope E0226 00:15:31.912887 1 reflector.go:199] github.com/containous/traefik/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: endpoints is forbidden: User "system:serviceaccount:kube-system:default" cannot list endpoints at the cluster scope

2、traefik的deployment文件

# cat traefik-deployment.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- kind: Deployment apiVersion: extensions/v1beta1 metadata: name: traefik-ingress-controller namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: replicas: 2 selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller hostNetwork: true nodeSelector: traefik: proxy terminationGracePeriodSeconds: 60 containers: - image: traefik name: traefik-ingress-lb ports: - name: web containerPort: 80 hostPort: 80 - name: admin containerPort: 8081 args: - --web - --web.address=:8081 - --kubernetes

3、traefik的service文件

# cat traefik-service.yaml apiVersion: v1 kind: Service metadata: name: traefik-web-ui namespace: kube-system spec: selector: k8s-app: traefik-ingress-lb ports: - port: 80 targetPort: 8081

4、通过yaml文件创建clusterrole、clusterrolebinding、deployment、serviceaccount、service

# ls # kubectl create -f traefik-rbac.yaml # kubectl create -f traefik-deployment.yaml # kubectl create -f traefik-service.yaml

# kubectl get pod -n kube-system # kubectl get svc -n kube-system # kubectl get svc

可以看到集群中default namespace中存在一个frontend服务。kube-system namespace中存在nginx-test、traefik-web-ui、kubernetes-dashboard三个服务。我们后续将创建4个ingress

通过web-ui可以看到在两个节点上各运行了一个pod

三、通过yaml文件创建ingress

# cat ui.yaml apiVersion: v1 kind: Service metadata: name: traefik-web-ui namespace: kube-system spec: selector: k8s-app: traefik-ingress-lb ports: - port: 80 targetPort: 8081 --- apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-web-ui namespace: kube-system annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: traefik-ui http: paths: - backend: serviceName: traefik-web-ui servicePort: 80# cat webui-ing.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-ingress namespace: kube-system annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: k8s.webui http: paths: - backend: serviceName: kubernetes-dashboard servicePort: 443# cat Redis-ing.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-ingress namespace: default annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: k8s.frontend http: paths: - backend: serviceName: frontend servicePort: 80# cat nginx-ing.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: traefik-nginx-ingress namespace: kube-system annotations: kubernetes.io/ingress.class: traefik spec: rules: - host: test.fjhb.cn http: paths: - backend: serviceName: nginx-test servicePort: 80# kubectl create -f ui.yaml # kubectl create -f webui-ing.yaml # kubectl create -f redis-ing.yaml # kubectl get ingress # kubectl get ingress -n kube-system