Kubernetes集群安全配置

这两天在梳理Kubernetes集群的安全配置，涉及到各个组件的配置，最终决定画一个图来展现，应该会更清晰。

这里写图片描述

涉及以下配置：

1. 其他各个组件作为client，访问kube-apiserver时，各个组件的配置，参考图中黑色线条对应的配置： - **kube-apiserver** ``` --secure-port=443 --client_ca_file=/var/run/kubernetes/dd_ca.crt --tls-private-key-file=/var/run/kubernetes/dd_server.key ```
   • kube-controller-manager ``` --kubeconfig=/etc/kubernetes/cmkubeconfig apiVersion: v1 kind: Config users - name: controllermanager user: client-certificate: /var/run/kubernetes/dd_cs_client.crt client-key: /var/run/kubernetes/dd_cs_client.key clusters: - name: local cluster: certificate-authority: /var/run/kubernetes/dd_ca.crt contexts: - context: cluster: local user: controllermanager name: my-context current-context: my-context ```
   • kube-scheduler kube-scheduler访问apiserver的安全配置同kube-controller-manager。
   • kubelet --kubeconfig=/var/lib/kubelet/kubeconfig apiVersion: v1 kind: Config users: - name: kubelet user: client-certificats: /home/dd_kubelet_client.crt client-key: /home/dd_kubelet_client.key clusters: - name: local cluster: certificate-authority: /home/dd_ca.crt contexts: - context: cluster: local user: kubelet name: my-context current-context: my-context
   • kube-proxy --kubeconfig=/var/lib/kubeproxy/proxykubeconfig apiVersion: v1 kind: Config users: - name: kubeproxy user: client-certificate: /home/dd_kubelet_client.crt client-key: /home/dd_kubelet_client.key clusters: - name: local cluster: certificate-authority: /home/dd_ca.crt contexts: - context: cluster: local user: kubeproxy name: my-context current-context: my-context
2. kube-apiserver作为client，访问kubelet server时的配置，参考图中绿色线条对应的配置：
   • kube-apiserver --kubelet-https --kubelet-certificate-authority=/var/run/kubelet/kubelet-ca.crt --kubelet-client-certificate=/var/run/kubelet/apiserver-kubelet.crt --kubelet-client-key=/var/run/kubelet/apiserver-kubelet.key
   • kubelet --client-ca-file=/var/run/kubelet/kubelet_ca.crt --tls-private-key-file=/var/run/kubelet/server.key --tls-cert-file string=/var/run/kubelet/server.crt
3. Pod访问kube-apiserver，是通过ServiceAccount来提供Token的, 涉及的配置见粉红色线条对应的内容。
   • Pod.Spec

   每个namespace都有一个default ServiceAccount。如果Pod.Spec.serivceAccountName未设置，这默认用default ServiceAccount。上图中的配置中，给Pod指明了一个自定义的Pod.Spec.serivceAccountName:build-rebot，automountServiceAccountToken: true表示自动将该ServiceAccount中的Secret定义的token,ca.crt,namespace挂载到Pod每个container内的以下对应目录： ServiceAccount Admission Make Sure Secret Volume Mounted:

/var/run/secrets/kubernetes.io/serviceaccount/token /var/run/secrets/kubernetes.io/serviceaccount/ca.crt /var/run/secrets/kubernetes.io/serviceaccount/namespace ```

- **kube-controller-manager**

    ```
    --root-ca-file=/var/run/kubernetes/dd_ca.crt 
    --service-account-private-key-file=/var/run/kubernetes/dd_server.key
    ```

这样Pod内的应用就能通过以下两种方式访问apiserver了：

- 添加kubectl proxy container，示例见[kubectl-container](https://github.com/kubernetes/kubernetes/tree/master/examples/kubectl-container/)

- use the Go client library, and create a client using the rest.InClusterConfig() and kubernetes.NewForConfig() functions. They handle locating and authenticating to the apiserver. [example](https://github.com/kubernetes/client-go/blob/master/examples/in-cluster/main.go)

4. kube-apiserver作为client，通过TLS访问etcd对应的配置见图中蓝色线条对应的内容。

- **kube-apiserver**

    ```
    --kubelet-https
    --kubelet-certificate-authority=/var/run/kubelet/etcd-ca.crt  
    --kubelet-client-certificate=/var/run/kubelet/etcd-kubelet.crt
    --kubelet-client-key=/var/run/kubelet/etcd-kubelet.key
    ```
- **etcd**

    ```
    --client-cert-auth 
    --trusted-ca-file=/etc/ssl/etcd/etcd-ca.crt 
    --cert-file=/etc/ssl/etcd/server.crt 
    --key-file=/etc/ssl/etcd/server.key
    ```

1. apiserver的Authentication Config：
   • kube-apiserver 以下三个flag，分别表示enable apiserver的x509 client certs, static token, static password三种认证方式。 --client-ca-file=/var/run/kubernetes/dd_ca.crt --token-auth-file=SOMEFILE --basic-auth-file=SOMEFILE

   其中token-auth-file对应文件内容格式为： ``` token1,user1,uid1,”group1,group2,group3" token2,user2,uid2,”group1,group2" ``` basic-auth-file对应文件内容格式为： ``` password1,user1,uid1,”group1,group2,group3" password2,user2,uid2,”group1,group2,group3" ```

2. apiserver的Authorization Config：
   • kube-apiserver 当前我们的环境中，使用默认值AlwaysAllow，如果有需要，后续会考虑enable RBAC。 --authorization-mode=AlwaysAllow
3. apiserver的Admission Control Config：
   • kube-apiserver 使用官方推荐的，v1.6+之后的配置为： --admission-control=NamespaceLifecycle, LimitRanger, ServiceAccount, PersistentVolumeLabel, DefaultStorageClass, ResourceQuota, DefaultTolerationSeconds