4. 死磕 k8s系列之安装包管理工具(Helm)
注：本文基于k8s集群v1.16.2版本。
注：如无特殊说明，以下所有操作都是在master节点上执行。
简介
Helm可以看作是k8s集群的包管理工具，通过Helm可以快速安装很多软件，比如mysql，nginx等，当然，也可以把自己的应用交给Helm来管理和安装。
Helm架构由Helm客户端、Helm服务端Tiller和Helm仓库Chart组成。
其中，Helm客户端可以部署在任意地方，只要能够访问k8s集群即可; Tiller服务端部署在k8s集群中。
注：为了方便，本文直接把Helm客户端安装在master节点上。
前提
必须有一个k8s集群。
安装Helm客户端
（1）下载Helm发布包
打开Helm Release，下载一个稳定版本，我这里下载的是v2.15.2版本的 Linuxamd64。
注：这里要下载稳定版本，不要下载RC（Release Candidate）版本，因为我们后面要修改镜像，非稳定版本国内的镜像仓库不一定有。
下载下来之后，解压，然后把helm可执行文件拷贝到/usr/local/bin目录下，之后就可以执行helm命令了。
[root@instance-ji0xk9uh-1 ~]# ls
calico-3.9.2.yaml helm-v2.15.2-linux-amd64.tar.gz init_master.sh install_kubelet.sh kubeadm-config.yaml nginx-ingress.yaml
[root@instance-ji0xk9uh-1 ~]# tar zxvf helm-v2.15.2-linux-amd64.tar.gz
linux-amd64/
linux-amd64/tiller
linux-amd64/helm
linux-amd64/README.md
linux-amd64/LICENSE
[root@instance-ji0xk9uh-1 ~]# ls
calico-3.9.2.yaml helm-v2.15.2-linux-amd64.tar.gz init_master.sh install_kubelet.sh kubeadm-config.yaml linux-amd64 nginx-ingress.yaml
[root@instance-ji0xk9uh-1 ~]# cp linux-amd64/helm /usr/local/bin/
[root@instance-ji0xk9uh-1 ~]# helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Error: could not find tiller安装tiller服务端
直接执行 helm init即可。
[root@master ~]# helm init
Creating /root/.helm
Creating /root/.helm/repository
Creating /root/.helm/repository/cache
Creating /root/.helm/repository/local
Creating /root/.helm/plugins
Creating /root/.helm/starters
Creating /root/.helm/cache/archive
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation检查是否安装成功， heml version。
[root@master ~]# helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Error: could not find a ready tiller pod发现并没有安装成功，再看看pod的状态， kubectlgetpod-n kube-system。
[root@master ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
# 省略...
tiller-deploy-58f57c5787-v6z7d 0/1 ImagePullBackOff 0 5m50s发现pod的状态为 ImagePullBackOff，出现这个的原因很大可能是镜像没有拉取下来。
我们使用 describe查看更详细的原因。
[root@master ~]# kubectl describe pod/tiller-deploy-58f57c5787-v6z7d -n kube-system
# 省略...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned kube-system/tiller-deploy-58f57c5787-v6z7d to node1
Normal Pulling 2m19s (x4 over 4m23s) kubelet, node1 Pulling image "gcr.io/kubernetes-helm/tiller:v2.15.2"
Warning Failed 2m4s (x4 over 4m7s) kubelet, node1 Failed to pull image "gcr.io/kubernetes-helm/tiller:v2.15.2": rpc error: code = Unknown desc = Error response from daemon: Get https://gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning Failed 2m4s (x4 over 4m7s) kubelet, node1 Error: ErrImagePull
Normal BackOff 97s (x6 over 4m7s) kubelet, node1 Back-off pulling image "gcr.io/kubernetes-helm/tiller:v2.15.2"
Warning Failed 84s (x7 over 4m7s) kubelet, node1 Error: ImagePullBackOff从这里可以看到它使用的镜像是 gcr.io/kubernetes-helm/tiller:v2.15.2，我们直接在镜像仓库搜索这个镜像。
[root@master ~]# docker search gcr.io/kubernetes-helm/tiller:v2.15.2
Error response from daemon: invalid registry endpoint https://gcr.io/v1/: Get https://gcr.io/v1/_ping: dial tcp 108.177.97.82:443: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry gcr.io` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/gcr.io/ca.crt从上面的日志可以看到是超时了，这时候我们换一个角度，看看我们的镜像仓库有没有 tiller的镜像。
[root@master ~]# docker search tiller
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
jessestuart/tiller Nightly multi-architecture (amd64, arm64, ar… 16 [OK]
sapcc/tiller Mirror of https://gcr.io/kubernetes-helm/til… 8
ist0ne/tiller https://gcr.io/kubernetes-helm/tiller 3 [OK]
rancher/tiller 2
jmgao1983/tiller from gcr.io/kubernetes-helm/tiller 2 [OK]
ibmcom/tiller Docker Image for IBM Cloud private-CE (Commu… 1
itinerisltd/tiller 1
luxas/tiller 1
ansibleplaybookbundle/tiller-apb An APB that deploys tiller for use with helm. 1 [OK]
cfplatformeng/tiller 0
cfplatformeng/tiller-ubuntu 0
ibmcom/tiller-ppc64le Docker Image for IBM Cloud Private-CE (Commu… 0
kubeapps/tiller-proxy A web-based UI for deploying and managing ap… 0
kubeapps/tiller-proxy-ci Store temporary images generated by CI system 0
sorenmat/tiller 0
cgetzen/tiller Custom Tiller Tests 0
appscode/tiller 0
fkpwolf/tiller 0
itinerisltd/tiller-circleci 0
anjia0532/tiller 0
pcanham/tiller tiller image for Raspberry Pi for testing He… 0
kontenapharos/tiller 0
renaultdigital/tillerless-helm-gcloud Add tillerless plugin to helm-gcloud image 0
4admin2root/tiller gcr.io/kubernetes-helm/tiller 0 [OK]
zhaowenlei/tiller FROM gcr.io/kubernetes-helm/tiller:TAG 0可以查到很多tiller的镜像，我们这里选第二个，看它的描述，是 gcr.io/kubernetes-helm/tiller的镜像。
我们尝试拉取看看，注意这里一定要拉取跟上面安装的helm相同版本的镜像。
[root@master ~]# docker pull sapcc/tiller:v2.15.2
v2.15.2: Pulling from sapcc/tiller
89d9c30c1d48: Pull complete
45d707e102b0: Pull complete
d20e148dce16: Pull complete
8ca326d5a0c5: Pull complete
Digest: sha256:4554a65fb8278d93f1c2c1f335ddbfcd6faa016c24b97e8de46c6b8fc1e9e7f5
Status: Downloaded newer image for sapcc/tiller:v2.15.2可以看到拉取成功了。所以，我们换一种思路，把tiller安装的yaml中使用的镜像改成上面这个可用的镜像，那么要怎么修改呢？
其实，也很简单，k8s提供了命令可以直接修改现在正在运行的pod的yaml配置， kubectl edit pod xxx。
注，这里也可以修改deployment的配置，这样更持久，因为pod是由deployment创建的。
[root@master ~]# kubectl edit pod tiller-deploy-58f57c5787-v6z7d -n kube-system
#省略...
spec:
automountServiceAccountToken: true
containers:
- env:
- name: TILLER_NAMESPACE
value: kube-system
- name: TILLER_HISTORY_MAX
value: "0"
image: sapcc/tiller:v2.15.2
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /liveness
port: 44135
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: tiller
#省略...找到image字段，把后面的镜像修改为 sapcc/tiller:v2.15.2。
然后，pod会自动更新，过一会再次查看pod的状态。
[root@master ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
#省略...
tiller-deploy-58f57c5787-v6z7d 1/1 Running 0 25m可以看到tiller运行起来了，再次使用 helm version检查结果。
[root@master ~]# helm version
Client: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.15.2", GitCommit:"8dce272473e5f2a7bf58ce79bb5c3691db54c96b", GitTreeState:"clean"}至此，说明tiller安装成功了。
使用helm安装mysql
helm search mysql，查找mysql的chart。
helm install mysql，安装mysql。
[root@master ~]# helm search mysql
NAME CHART VERSION APP VERSION DESCRIPTION
stable/mysql 1.4.0 5.7.27 Fast, reliable, scalable, and easy to use open-source rel...
stable/mysqldump 2.6.0 2.4.1 A Helm chart to help backup MySQL databases using mysqldump
stable/prometheus-mysql-exporter 0.5.2 v0.11.0 A Helm chart for prometheus mysql exporter with cloudsqlp...
stable/percona 1.2.0 5.7.17 free, fully compatible, enhanced, open source drop-in rep...
stable/percona-xtradb-cluster 1.0.3 5.7.19 free, fully compatible, enhanced, open source drop-in rep...
stable/phpmyadmin 4.1.1 4.9.1 phpMyAdmin is an mysql administration frontend
stable/gcloud-sqlproxy 0.6.1 1.11 DEPRECATED Google Cloud SQL Proxy
stable/mariadb 6.12.2 10.3.18 Fast, reliable, scalable, and easy to use open-source rel...
[root@master ~]# helm install stable/mysql
Error: no available release name found又报错了，经查找相关资料，是说k8s集群1.16.x版本加入了RBAC权限相关的东西，必须给tiller定义一个service account。
OK，使用以下脚本，先卸载tiller（删除相应的deployment），再设置权限，重新安装。
注，如果安装tiller过程中出现如下错误也是执行下面的脚本。
Error:error installing:the server couldnotfind the requested resource
重装tiller脚本。
# 先卸载tiller，删除其deployment即可
kubectl delete deployment.apps/tiller-deploy -n kube-system
# 创建serviceaccount
kubectl create serviceaccount --namespace kube-system tiller
# 绑定
kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
# 重新初始化tiller
helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | kubectl apply -f -执行结果如下：
[root@master ~]# kubectl get deployment -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
calico-kube-controllers 1/1 1 1 22h
coredns 2/2 2 2 22h
kuboard 1/1 1 1 21h
tiller-deploy 1/1 1 1 7m21s
[root@master ~]# kubectl delete deployment.apps/tiller-deploy -n kube-system
deployment.apps "tiller-deploy" deleted
[root@master ~]# kubectl create serviceaccount --namespace kube-system tiller
ec.selector.matchLabels.'app'='helm' --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | kubectl apply -f -serviceaccount/tiller created
[root@master ~]# kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller created
[root@master ~]# helm init --service-account tiller --override spec.selector.matchLabels.'name'='tiller',spec.selector.matchLabels.'app'='helm' --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | kubectl apply -f -
deployment.apps/tiller-deploy created
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
service/tiller-deploy configured这时候又会出现 ImagePullBackOff错误，同样地，重新修改tiller的yaml文件中的镜像，会自动更新。
重新安装后，再重新安装mysql。
[root@master ~]# helm install stable/mysql
NAME: bumptious-fish
LAST DEPLOYED: Thu Oct 31 15:21:43 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
bumptious-fish-mysql-test 1 0s
==> v1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
bumptious-fish-mysql 0/1 1 0 0s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
bumptious-fish-mysql Pending 0s Filesystem
==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
bumptious-fish-mysql-68bf985647-649ms 0/1 Pending 0 0s
==> v1/Secret
NAME TYPE DATA AGE
bumptious-fish-mysql Opaque 2 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
bumptious-fish-mysql ClusterIP 10.96.43.224 <none> 3306/TCP 0s
NOTES:
MySQL can be accessed via port 3306 on the following DNS name from within your cluster:
bumptious-fish-mysql.default.svc.cluster.local
To get your root password run:
MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default bumptious-fish-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
To connect to your database:
1. Run an Ubuntu pod that you can use as a client:
kubectl run -i --tty ubuntu --image=ubuntu:16.04 --restart=Never -- bash -il
2. Install the mysql client:
$ apt-get update && apt-get install mysql-client -y
3. Connect using the mysql cli, then provide your password:
$ mysql -h bumptious-fish-mysql -p
To connect to your database directly from outside the K8s cluster:
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
# Execute the following command to route the connection:
kubectl port-forward svc/bumptious-fish-mysql 3306
mysql -h ${MYSQL_HOST} -P${MYSQL_PORT} -u root -p${MYSQL_ROOT_PASSWORD}查看mysql是否启动成功，我们是安装到默认命名空间下的，所以不需要带 -n参数了。
[root@master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
bumptious-fish-mysql-68bf985647-649ms 0/1 Pending 0 40s会发现一直是在 Pending状态，同样地，使用 kubectl describe命令查看详细的错误。
[root@master ~]# kubectl describe pod bumptious-fish-mysql-68bf985647-649ms
# 省略...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 2 times)
Warning FailedScheduling <unknown> default-scheduler pod has unbound immediate PersistentVolumeClaims (repeated 2 times)看最后两行，是说pod没有绑定到PersistentVolumeClaims（简写pvc），我们再查查pvc。
[root@master ~]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
bumptious-fish-mysql Pending 2m18s可以看到pvc也是 Pending状态，同样地，describe一下。
[root@master ~]# kubectl describe pvc bumptious-fish-mysql
# 省略...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal FailedBinding 11s (x12 over 2m44s) persistentvolume-controller no persistent volumes available for this claim and no storage class is set看最后一行，说是没有可用的persistent volume（简称pv）且没有设置storage class，这又是些什么东西呢。
这里就牵涉到k8s的整体架构了，针对有状态的pod，需要先关联一个pvc，pvc会与具体的pv关联，pv可以理解成是磁盘上的一块空间，或者是远程存储等。
关于pv的自动创建也有很多方案，比如nfs等，为了不增加复杂性，我们这里采用手动创建两块pv给mysql使用。
手动创建pv的脚本如下：
kubectl create -f - <<EOF
kind: PersistentVolume
apiVersion: v1
metadata:
name: [名称]
labels:
type: local
spec:
capacity:
storage: [使用的空间]
accessModes:
- ReadWriteOnce
hostPath:
path: "存储的路径"
EOF执行结果如下：
[root@master ~]# kubectl create -f - <<EOF
> kind: PersistentVolume
> apiVersion: v1
> metadata:
> name: mysql-pv
> labels:
> type: local
> spec:
> capacity:
> storage: 10Gi
> accessModes:
> - ReadWriteOnce
> hostPath:
> path: "/data/storage/mysql"
> EOF
persistentvolume/mysql-pv created
[root@master ~]# kubectl get pv,pvc,pod
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/mysql-pv 10Gi RWO Retain Bound default/bumptious-fish-mysql 67s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/bumptious-fish-mysql Bound mysql-pv 10Gi RWO 5m49s
NAME READY STATUS RESTARTS AGE
pod/bumptious-fish-mysql-68bf985647-649ms 1/1 Running 0 5m49s这里的 kubectlgetpv,pvc,pod要过一会再执行，或者前面加一个watch，即 watch kubectlgetpv,pvc,pod，直到pod的状态变成Running且READY为1/1的时候表示mysql安装成功了。
到这里，基本可以确定mysql是安装成功了，但是没有连接进去谁又敢百分百说安装成功了呢，OK，我们下面尝试连接到mysql中。
连接mysql
查看mysql的service。
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
bumptious-fish-mysql ClusterIP 10.96.43.224 <none> 3306/TCP 9m36s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 23h可以看到服务暴露的方式为 ClusterIP，我们把它修改为 NodePort，NodePort可以让我们从外部访问服务。
[root@master ~]# kubectl edit svc bumptious-fish-mysql
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2019-10-31T07:21:43Z"
labels:
app: bumptious-fish-mysql
chart: mysql-1.4.0
heritage: Tiller
release: bumptious-fish
name: bumptious-fish-mysql
namespace: default
resourceVersion: "125451"
selfLink: /api/v1/namespaces/default/services/bumptious-fish-mysql
uid: 70be1da0-a10f-4439-9d3e-fd516f8785bf
spec:
clusterIP: 10.96.43.224
externalTrafficPolicy: Cluster
ports:
- name: mysql
nodePort: 32684
port: 3306
protocol: TCP
targetPort: mysql
selector:
app: bumptious-fish-mysql
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}找到spec.type，将其值修改为 NodePort，保存退出，查看自动生成的端口。
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
bumptious-fish-mysql NodePort 10.96.43.224 <none> 3306:32684/TCP 18h
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 41h可以看到生成的端口为 32684，再查看mysql的初始密码，命令在安装mysql的时候提示过，注意这里拷贝你安装时提示的命令。
[root@master ~]# echo $(kubectl get secret --namespace default bumptious-fish-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
3XvXkqckUE在外部使用mysql客户端连接（当然，也可以在集群内部装一个mysql客户端，这样就不需要暴露外网ip及端口了）：
ip为任意worker节点的公网ip；
端口为上面获取到的32684；
用户名为root，密码为上面获取到的3XvXkqckUE；
C:\> mysql -h[xx.xx.xx.xx] -P32684 -uroot -p3XvXkqckUE
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 13658
Server version: 5.7.14 MySQL Community Server (GPL)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.04 sec)
mysql> create database test;
Query OK, 1 row affected (0.04 sec)
mysql> use test;
Database changed
mysql> create table user(id int, name varchar(50));
Query OK, 0 rows affected (0.05 sec)
mysql> insert into user values(1,"hello");
Query OK, 1 row affected (0.05 sec)
mysql> select * from user;
+------+-------+
| id | name |
+------+-------+
| 1 | hello |
+------+-------+
1 row in set (0.04 sec)成功连接进去了，说明mysql安装成功了。
总结
本章成功安装了Helm，并使用Helm成功安装了mysql，这里有几个需要注意的点。
（1）k8s集群v1.16.x版本需要设置RBAC权限，才能成功安装Helm；
（2）网络问题导致无法成功拉取tiller镜像，所以需要手动修改tiller pod的yaml文件；
（3）mysql安装需要创建持久卷PV，我们这里使用手动创建的方式。
参考
官方文档：https://helm.sh/docs/using_helm
本文在官方文档的基础上补充了一些异常的处理。
本文参与 腾讯云自媒体分享计划，分享自微信公众号。
原始发表：2019-12-31，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
目录
相关产品与服务
云数据库 SQL Server
腾讯云数据库 SQL Server （TencentDB for SQL Server）是业界最常用的商用数据库之一，对基于 Windows 架构的应用程序具有完美的支持。TencentDB for SQL Server 拥有微软正版授权，可持续为用户提供最新的功能，避免未授权使用软件的风险。具有即开即用、稳定可靠、安全运行、弹性扩缩等特点。