二进制安装k8s集群(14)-安装coredns
在上一篇文章里我们主要介绍worker组件kube-proxy的安装，这里我们开始介绍安装k8s集群内的一些基础服务，所有的基础服务都创建在kube-system这个namesapce里，我们从coredns开始。coredns提供k8s集群内部service的fqdn服务，是以deployment的方式运行在k8s集群内部的。image镜像从我们的private repo pull下来(以前文章里介绍过harbor private repo的创建，以及镜像的push和pull)。当然原始image来源于官方的k8s.gcr.io/coredns:1.3.1，不过要下载它需要访问国外网站。
创建配置文件目录：
由于coredns是以deployment的方式部署在k8s集群里的，一般都会有yaml部署文件，目前都放在此目录里。
mkdir -p /opt/application/k8s/core-dns
cd /opt/application/k8s/core-dns
创建image pull secret：
由于我们使用的是private repo的private project里的image，所以k8s在pull image的时候需要repo的认证，这个认证信息就存储在k8s secret资源里。
kubectl create secret docker-registry container-registry --docker-server=172.20.11.41:1034 \
--docker-username=admin --docker-password=abc123_ --namespace=kube-system
kubectl describe secret container-registry -n kube-system
创建coredns的service-account：
将上个步骤创建的secret赋给这个service-account，另外coredns需要访问kube-apiserver来得到service name和cluster ip，从而建立fqdn服务。对于资源访问，k8s有自己的策略，这里给coredns创建单独的service-account，cluster-role，cluster-role-binding。这里就不详细展开，有兴趣的同学可以看一下k8s的RBAC访问策略。
cat > /opt/application/k8s/core-dns/coredns-service-account.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
name: serviceaccount-coredns
namespace: kube-system
imagePullSecrets:
- name: container-registry
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-service-account.yaml
kubectl describe serviceaccount serviceaccount-coredns -n kube-system
创建coredns的cluster-role：
cat > /opt/application/k8s/core-dns/coredns-cluster-role.yaml <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cluster-role-coredns
rules:
- apiGroups:
- ""
resources:
- endpoints
- services
- pods
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-cluster-role.yaml
kubectl describe clusterrole cluster-role-coredns
创建coredns的cluster-role-binding：
cat > /opt/application/k8s/core-dns/coredns-cluster-role-binding.yaml <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cluster-role-binding-coredns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-role-coredns
subjects:
- kind: ServiceAccount
name: serviceaccount-coredns
namespace: kube-system
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-cluster-role-binding.yaml
kubectl describe clusterrolebinding cluster-role-binding-coredns -n kube-system创建coredns的配置configmap：
coredns也有自己的配置，我们把它的配置创建在k8s configmap资源里，然后在容器里挂载这个configmap的数据，从而提供给coredns配置。当然配置项比较多，这里就不逐一介绍，有兴趣的同学请参考coredns的配置文档。
cat > /opt/application/k8s/core-dns/coredns-config-map.yaml <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap-coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health
kubernetes cluster.local {
endpoint https://172.20.11.41:6443
tls /etc/coredns/k8scert/k8sapiserver-client.crt /etc/coredns/k8scert/k8sapiserver-client.key /etc/coredns/k8scert/ca.crt
pods insecure
upstream
fallthrough cluster.local
ttl 30
}
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-config-map.yaml
kubectl describe configmap configmap-coredns -n kube-system
创建coredns的配置secret ：
coredns是需要与kube-apiserver交互的，我们的kube-apiserver开启了ssl验证，所以在coredns里也要配置相应的证书，这个请提前制作好(可以参考以前文章制作docker server证书)。对于k8s来说ssl证书可以存储在secret对象里(和configmap相比只是把原文本base64了一下)，然后挂载到容器中，给coredns提供配置。
kubectl create secret generic core-dns-k8s-access-secret --namespace=kube-system \
--from-file=k8sapiserver-client.key=/opt/sw/cert/k8sapiserver-client.key \
--from-file=k8sapiserver-client.crt=/opt/sw/cert/k8sapiserver-client.crt \
--from-file=ca.crt=/opt/sw/cert/ca.crt
kubectl describe secret core-dns-k8s-access-secret -n kube-system
创建coredns的deployment：
cat > /opt/application/k8s/core-dns/coredns-deployment.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-coredns
namespace: kube-system
labels:
k8s-app: kube-dns
spec:
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
spec:
serviceAccountName: serviceaccount-coredns
containers:
- name: coredns
image: 172.20.11.41:1034/infra/coredns:latest
imagePullPolicy: IfNotPresent
args: [ "-conf", "/etc/coredns/Corefile" ]
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
readOnly: true
- name: config-volume-k8s-cert
mountPath: /etc/coredns/k8scert
readOnly: true
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- all
readOnlyRootFilesystem: true
dnsPolicy: Default
volumes:
- name: config-volume
configMap:
name: configmap-coredns
items:
- key: Corefile
path: Corefile
- name: config-volume-k8s-cert
secret:
secretName: core-dns-k8s-access-secret
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-deployment.yaml
kubectl describe deployment deployment-coredns -n kube-system
创建coredns的service：
这个service会生成coredns在集群里的cluster-ip，这个cluster-ip在kubetel里配置，然后在kubelet创建容器的时候将这个ip配置为容器的dns服务器地址。所以请在配置文件的clusterIP字段里设置value和以前文章中kubelet的--cluster-dns配置项一致。
cat > /opt/application/k8s/core-dns/coredns-service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
name: service-core-dns
namespace: kube-system
labels:
k8s-app: kube-dns
spec:
selector:
k8s-app: kube-dns
clusterIP: 10.254.10.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
protocol: TCP
- name: metrics
port: 9153
protocol: TCP
EOF
kubectl create -f /opt/application/k8s/core-dns/coredns-service.yaml
kubectl describe service service-core-dns -n kube-system
查看coredns在集群中的pod：
kubectl get pods -n kube-system
kubectl logs deployment-coredns-c58b8b7fc-kp2j8 -n kube-system