kubernetes中其他控制器之PodSecurityPolicy
kubernetes中其他控制器之PodSecurityPolicy
PodSecurityPolicy是集群级别的Pod安全策略，自动为集群中的Pod和Volume设置Security Context。
Admission Controller（准入控制器）拦截对 kube-apiserver 的请求，拦截发生在请求的对象被持久化之前，但是在请求被验证和授权之后。这样我们就可以查看请求对象的来源，并验证需要的内容是否正确。通过将它们添加到 kube-apiserver 的--enable-admission-plugins参数中来启用准入控制器。所以如果我们要使用PSP，我们就需要在kube-apiserver中添加起参数，如下：
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodSecurityPolicy其他插件是kubernetes官方推荐的插件。
然后重启kube-apiserver：
# systemctl daemon-reload
# systemctl restart kube-apiserver这时候就已经启动了PSP控制器，如果我们现在创建Pod试试：
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent然后我们kubectl get pod的时候发现并没有pod。我们再查看deploy的状态，如下：
# kubectl get deployments.
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 0/1 0 0 117s
# kubectl get replicasets.
NAME DESIRED CURRENT READY AGE
nginx-55fc968d9 1 0 0 2m17s我们看到replicaset和deploy都创建成功了，但是replicaset并没有创建pod，这是因为我们集群现在缺少安全策略，所以创建新的Pod不会成功，这时我们就需要使用ServiceAccount。
正常情况下，我们并不会直接创建Pod，都是通过其他控制器比如Deployment、StatefulSet等来创建Pod。我们现在要使用PSP，需要配置kube-controller-manager来为其包含的每个控制器单独使用ServiceAccount，我们可以通过以下参数来添加，如下：
--use-service-account-credentials=true然后重启controller-manager：
# systemctl daemon-reload
# systemctl restart kube-controller-manager然后kubenetes就会自动生成如下一些SA，这些SA就指定了哪个控制器可以解析哪些策略：
# kubectl get serviceaccount -n kube-system | egrep -o '[A-Za-z0-9-]+-controller'
attachdetach-controller
certificate-controller
clusterrole-aggregation-controller
cronjob-controller
daemon-set-controller
deployment-controller
disruption-controller
endpoint-controller
expand-controller
job-controller
namespace-controller
node-controller
pv-protection-controller
pvc-protection-controller
replicaset-controller
replication-controller
resourcequota-controller
service-account-controller
service-controller
statefulset-controller
traefik-ingress-controller
ttl-controllerPSP提供一种声明式的方式，用于表达运行的用户和ServiceAccount在我们集群中创建的内容。其主要的策略有：
控制项 | 说明 |
|---|---|
privileged | 运行特权容器 |
defaultAddCapabilities | 可添加到容器的Capabilities |
requiredDropCapabilities | 会从容器中删除的Capabilities |
volumes | 控制容器可以使用哪些volume |
hostNetwork | host网络 |
hostPorts | 允许的host端口列表 |
hostPID | 使用host PID namespace |
hostIPC | 使用host IPC namespace |
seLinux | SELinux Context |
runAsUser | user ID |
supplementalGroups | 允许的补充用户组 |
fsGroup | volume FSGroup |
readOnlyRootFilesystem | 只读根文件系统 |
在上面的示例中，我们需要两个策略： 1、提供限制访问的默认策略，保证使用特权设置无法创建Pod； 2、提升许可策略，允许将特权设置用于某些Pod，比如允许在特定命名空间下创建Pod；
首先，创建一个默认策略： psp-restrictive.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: restrictive
spec:
privileged: false
hostNetwork: false
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
hostPID: false
hostIPC: false
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
volumes:
- 'configMap'
- 'downwardAPI'
- 'emptyDir'
- 'persistentVolumeClaim'
- 'secret'
- 'projected'
allowedCapabilities:
- '*'然后直接创建这个PSP对象：
# kubectl apply -f psp-restrictive.yaml
podsecuritypolicy.policy/restrictive created
# kubectl get psp
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
restrictive false * RunAsAny RunAsAny RunAsAny RunAsAny false configMap,downwardAPI,emptyDir,persistentVolumeClaim,secret,projected其次，创建一个提升策略，用于那些需要提升权限的Pod，比如kube-proxy，它就需要hostNetwork权限： psp-permissive.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: permissive
spec:
privileged: true
hostNetwork: true
hostIPC: true
hostPID: true
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
hostPorts:
- min: 0
max: 65535
volumes:
- '*'然后创建这个PSP对象：
# kubectl apply -f psp-permissive.yaml
podsecuritypolicy.policy/permissive created
# kubectl get psp
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
permissive true RunAsAny RunAsAny RunAsAny RunAsAny false *
restrictive false * RunAsAny RunAsAny RunAsAny RunAsAny false configMap,downwardAPI,emptyDir,persistentVolumeClaim,secret,projected但是仅仅部署了这两个策略是不够的，我们RBAC进行授权，不然我们的Pod还是不能创建成功。RBAC确定一个ServiceAccount可以使用的策略，如果使用ClusterRoleBinding可以为ServiceAccount提供限制性策略（restrictive）的访问，如果使用RoleBinding可以为SeriveAccount提供虚空策略的访问。
首先创建允许使用restrictive策略的ClusterRole，然后再创建一个ClusterRoleBinding将所有控制器的ServiceAccount进行绑定： psp-restrictive-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp-restrictive
rules:
- apiGroups:
- extensions
resources:
- podsecruritypolicies
resourceNames:
- restrictive
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: psp-default
subjects:
- kind: Group
name: system:serviceaccounts
namespace: kube-system
roleRef:
kind: ClusterRole
name: psp-restrictive
apiGroup: rbac.authorization.k8s.io然后创建RBAC资源对象：
# kubectl apply -f psp-restrictive-rbac.yaml
clusterrole.rbac.authorization.k8s.io/psp-restrictive created
clusterrolebinding.rbac.authorization.k8s.io/psp-default created然后我们可以看到外面刚开始创建的Pod现在可以创建了：
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-55fc968d9-qn2vr 1/1 Running 0 17m但是如果我们现在给这个Deployment清单加一个hostNetwork=true这个特权，观察Pod是否能够创： nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
hostNetwork: true然后我们再次创建这个Deployment：
# kubectl apply -f nginx-deploy.yaml
deployment.apps/nginx created
# kubectl get pod
NAME READY STATUS RESTARTS AGE
# kubectl get deployments.
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 0/1 0 0 12s
# kubectl get replicasets.
NAME DESIRED CURRENT READY AGE
nginx-5cd65fd4c6 1 0 0 18s我们可以看到Pod并未被创建，我们describe一个replicaset，发现如下日志：
# kubectl describe rs nginx-5cd65fd4c6
......
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 41s (x16 over 3m24s) replicaset-controller Error creating: pods "nginx-5cd65fd4c6-" is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]提示我们hostNetwork不允许被使用。
但是在某些情况下，我们需要在某个命名空间下使用特权，这时候我们就可以创建一个允许使用特权的ClusterRole，但是这里我们为特定ServiceAccount设置RoleBinding，如下： psp-permissive-rbac.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: psp-permissive
rules:
- apiGroups:
- extensions
resources:
- podsecuritypolicies
resourceNames:
- permissive
verbs:
- use
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: psp-permissive
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp-permissive
subjects:
- kind: ServiceAccount
name: daemon-set-controller
namespace: kube-system
- kind: ServiceAccount
name: replicaset-controller
namespace: kube-system
- kind: ServiceAccount
name: job-controller
namespace: kube-system上面定义了对kube-system中的daemonset，replicaset，job拥有特权创建Pod。 然后我们创建RBAC资源清单：
# kubectl apply -f psp-permissive-rbac.yaml
clusterrole.rbac.authorization.k8s.io/psp-permissive created
rolebinding.rbac.authorization.k8s.io/psp-permissive created现在我们定义一个测试Deployment：
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
hostNetwork: true然后创建资源对象：
# kubectl apply -f nginx-deploy.yaml
deployment.apps/nginx created
[root@ecs-5704-0003 kubernetes]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
......
nginx-5cd65fd4c6-7pn9z 1/1 Running 0 4s然后我们可以看到Pod可以正常被创建。
另外还有一种特殊的需求，就是在某个命令空间下只有某个应用可以使用特权，那么针对这类应用就需要单独创建一个SA，然后和permissive策略镜像RoleBinding了，如下： （1）、创建可以使用特权的SA
# kubectl create serviceaccount specialsa
serviceaccount/specialsa created（2）、创建RoleBinding specialsa-psp.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: specialsa-psp-permissive
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: psp-permissive
subjects:
- kind: ServiceAccount
name: specialsa
namespace: default然后创建上面的RoleBinding对象：
# kubectl apply -f specialsa-psp.yaml
rolebinding.rbac.authorization.k8s.io/specialsa-psp-permissive created然后创建一个测试的Deployment： ng-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-hostnetwork-deploy
namespace: default
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
hostNetwork: true
serviceAccount: specialsa # 注意这里使用的sa的权限绑定然后创建资源对象：
# kubectl apply -f ng-deploy.yaml
deployment.apps/nginx-hostnetwork-deploy created
# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginx-hostnetwork-deploy-7b65cf7bbd-g5wl5 1/1 Running 0 2s然后可以发现在default命名空间下可以创建拥有特权的Pod了。