k8s1.5.4挂载volume之nfs
volume的例子集合
https://github.com/kubernetes/kubernetes/tree/master/examples/volumes
http://www.dockerinfo.net/2926.html
https://kubernetes.io/docs/user-guide/volumes/
其他相关文档
k8s集群安装部署
http://jerrymin.blog.51cto.com/3002256/1898243
k8s集群RC、SVC、POD部署
http://jerrymin.blog.51cto.com/3002256/1900260
k8s集群组件kubernetes-dashboard和kube-dns部署
http://jerrymin.blog.51cto.com/3002256/1900508
k8s集群监控组件heapster部署
http://jerrymin.blog.51cto.com/3002256/1904460
k8s集群反向代理负载均衡组件部署
http://jerrymin.blog.51cto.com/3002256/1904463
k8s集群挂载volume之nfs
http://jerrymin.blog.51cto.com/3002256/1906778
k8s集群挂载volume之glusterfs
http://jerrymin.blog.51cto.com/3002256/1907274
参考github上的例子
[root@k8s-master nfs]# pwd
/usr/local/kubernetes/examples/volumes/nfs
[root@k8s-master nfs]# cat README.md
## Quickstart
```console
$ kubectl create -f examples/volumes/nfs/provisioner/nfs-server-gce-pv.yaml
$ kubectl create -f examples/volumes/nfs/nfs-server-rc.yaml
$ kubectl create -f examples/volumes/nfs/nfs-server-service.yaml
# get the cluster IP of the server using the following command
$ kubectl describe services nfs-server
# use the NFS server IP to update nfs-pv.yaml and execute the following
$ kubectl create -f examples/volumes/nfs/nfs-pv.yaml
$ kubectl create -f examples/volumes/nfs/nfs-pvc.yaml
# run a fake backend
$ kubectl create -f examples/volumes/nfs/nfs-busybox-rc.yaml
# get pod name from this command
$ kubectl get pod -l name=nfs-busybox
# use the pod name to check the test file
$ kubectl exec nfs-busybox-jdhf3 -- cat /mnt/index.html
```
具体操作
[root@k8s-master nfs]# kubectl create -f provisioner/nfs-server-gce-pv.yaml
persistentvolumeclaim "nfs-pv-provisioning-demo" created
刚添加的PVC的状态是Pending,如果有合适的PV,这个Pending状态会立刻变为Bound,同时相应的PVC也会变为Bound。 你也可以先添加PVC,后添加PV,这样就能保证看到Pending状态。
[root@k8s-master nfs]# kubectl create -f nfs-server-rc.yaml
The ReplicationController "nfs-server" is invalid: spec.template.spec.containers[0].securityContext.privileged: Forbidden: disallowed by policy
查找参数--allow-privileged为true后k8s将允许在pod中运行拥有系统特权的容器应用
修改/etc/kubernetes/config 值KUBE_ALLOW_PRIV="--allow-privileged=true"后重启所有组件
但是这种方式出现了错误,错误见https://github.com/kubernetes/kubernetes/issues/43120
后调整了方案:
nfs服务器不在容器里部署,直接在节点上部署,然后容器挂载的方式测试,毕竟生存环境中存储一般也不会跑在容器里,开始图方便直接用nfs容器了。nfs服务器搭建比较简单,这里省略了。可以参考http://www.cnblogs.com/zhangmingcheng/p/6134210.html。
[root@k8s-master nfs]# vim /etc/exports
[root@k8s-master nfs]# systemctl enable rpcbind.service
[root@k8s-master nfs]# systemctl enable nfs-server.service
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@k8s-master nfs]# systemctl start rpcbind.service
[root@k8s-master nfs]# systemctl start nfs-server.service
[root@k8s-master nfs]# rpcinfo -p
program vers proto port service
100000 4 tcp 111 portmapper
100000 3 tcp 111 portmapper
100000 2 tcp 111 portmapper
100000 4 udp 111 portmapper
100000 3 udp 111 portmapper
100000 2 udp 111 portmapper
100024 1 udp 37284 status
100024 1 tcp 57305 status
100005 1 udp 20048 mountd
100005 1 tcp 20048 mountd
100005 2 udp 20048 mountd
100005 2 tcp 20048 mountd
100005 3 udp 20048 mountd
100005 3 tcp 20048 mountd
100003 3 tcp 2049 nfs
100003 4 tcp 2049 nfs
100227 3 tcp 2049 nfs_acl
100003 3 udp 2049 nfs
100003 4 udp 2049 nfs
100227 3 udp 2049 nfs_acl
100021 1 udp 36397 nlockmgr
100021 3 udp 36397 nlockmgr
100021 4 udp 36397 nlockmgr
100021 1 tcp 40459 nlockmgr
100021 3 tcp 40459 nlockmgr
100021 4 tcp 40459 nlockmgr
主要node节点, flanneld,docker网络都要有挂载权限才行
[root@k8s-master nfs]# exportfs
/data/nfs 10.1.0.0/16
/data/nfs 10.254.0.0/16
/data/nfs 172.17.3.0/24
[root@k8s-master nfs]# showmount -e
Export list for k8s-master:
/data/nfs 172.17.3.0/24,10.254.0.0/16,10.1.0.0/16
[root@k8s-master nfs]# vim nfs-pv.yaml
nfs:
# FIXME: use the right IP
server: 172.17.3.20
path: "/data/nfs"
[root@k8s-master nfs]# kubectl create -f nfs-pv.yaml
persistentvolume "nfs" created
[root@k8s-master nfs]# kubectl create -f nfs-pvc.yaml
persistentvolumeclaim "nfs" created
[root@k8s-master nfs]# kubectl create -f nfs-web-rc.yaml
replicationcontroller "nfs-web" created
[root@k8s-master nfs]# kubectl create -f nfs-web-service.yaml
service "nfs-web" created
查看PV\PVC状态
[root@k8s-master nfs]# kubectl get pv
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
nfs 100Mi RWX Retain Bound default/nfs 25m
[root@k8s-master nfs]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES AGE
nfs Bound nfs 100Mi RWX 25m
进入容器验证
[root@k8s-master ~]# kubectl get pods |grep nfs-web
nfs-web-gj1qr 1/1 Running 0 7m
nfs-web-vrzh4 1/1 Running 0 8m
root@nfs-web-vrzh4:/usr/share/nginx/html# df -h |grep nginx
172.17.3.20:/data/nfs 422G 925M 421G 1% /usr/share/nginx/html
[root@k8s-master ~]# cd /data/nfs/
[root@k8s-master nfs]# echo 'hello world!' > index.html
root@nfs-web-vrzh4:/usr/share/nginx/html# cat index.html
hello world!
[root@k8s-master nfs]# kubectl get ep|grep nfs-web
nfs-web 10.1.15.2:80,10.1.39.11:80 15m
[root@k8s-master nfs]# curl 10.1.15.2:80
hello world!
[root@k8s-master nfs]# curl 10.1.39.11:80
hello world!