持续集成Harbor+Helm

原创

ruochen

发布于 2021-11-26 13:18:15

1.2K0

发布于 2021-11-26 13:18:15

文章被收录于专栏：若尘的技术专栏

1 环境准备

在原来Harbor 2.1.2安装的基础上，继续集成Helm仓库

Helm 是一个命令行下的客户端工具。主要用于 Kubernetes 应用程序 Chart 的创建、打包、发布以及创建和管理本地和远程的 Chart 仓库。
Tiller 是 Helm 的服务端，部署在 Kubernetes 集群中。Tiller 用于接收 Helm 的请求，并根据 Chart 生成 Kubernetes 的部署文件（ Helm 称为 Release ），然后提交给 Kubernetes 创建应用。Tiller 还提供了 Release 的升级、删除、回滚等一系列功能。
Chart Helm 的软件包，采用 TAR 格式。类似于 APT 的 DEB 包或者 YUM 的 RPM 包，其包含了一组定义 Kubernetes 资源相关的 YAML 文件
Repoistory Helm 的软件仓库，Repository 本质上是一个 Web 服务器，该服务器保存了一系列的 Chart 软件包以供用户下载，并且提供了一个该 Repository 的 Chart 包的清单文件以供查询。Helm 可以同时管理多个不同的 Repository。-->Harbor
Release 使用 helm install 命令在 Kubernetes 集群中部署的 Chart 称为 Release

注意点：

helm3移除了tiller这个组件，默认通过~/.kube/config与集群进行交互，也就是说使用了与kubctl相同的上下文访问权限，若不在默认位置可通过–kubeconfig参数进行指定，按照官方安装文档安装即可直接使用

**-------------------------------Harbor服务器

相关操作如下----------------------------**

2 启用 Harbor的chart repository服务

默认新版 harbor不会启用 chart repository service ，如果需要管理 helm ，我们需要在安装时添加额外的参数

## 默认安装

$ cd /usr/local/harbor

$ ./install.sh

## 启动 chart repository service 服务

$ cd /usr/local/harbor

$ ./install.sh --with-chartmuseum

等待安装完成，完成后会有如下提示：

[Step 0]: checking if docker is installed ...

Note: docker version: 20.10.1

[Step 1]: checking docker-compose is installed ...

Note: docker-compose version: 1.27.4

[Step 2]: loading Harbor images ...

Loaded image: goharbor/chartmuseum-photon:v2.1.2

Loaded image: goharbor/prepare:v2.1.2

Loaded image: goharbor/harbor-log:v2.1.2

Loaded image: goharbor/harbor-registryctl:v2.1.2

Loaded image: goharbor/clair-adapter-photon:v2.1.2

Loaded image: goharbor/harbor-db:v2.1.2

Loaded image: goharbor/harbor-jobservice:v2.1.2

Loaded image: goharbor/clair-photon:v2.1.2

Loaded image: goharbor/notary-signer-photon:v2.1.2

Loaded image: goharbor/harbor-portal:v2.1.2

Loaded image: goharbor/redis-photon:v2.1.2

Loaded image: goharbor/nginx-photon:v2.1.2

Loaded image: goharbor/trivy-adapter-photon:v2.1.2

Loaded image: goharbor/harbor-core:v2.1.2

Loaded image: goharbor/registry-photon:v2.1.2

Loaded image: goharbor/notary-server-photon:v2.1.2

[Step 3]: preparing environment ...

[Step 4]: preparing harbor configs ...

prepare base dir is set to /usr/local/harbor

Clearing the configuration file: /config/portal/nginx.conf

Clearing the configuration file: /config/log/logrotate.conf

Clearing the configuration file: /config/log/rsyslog_docker.conf

Clearing the configuration file: /config/nginx/nginx.conf

Clearing the configuration file: /config/core/env

Clearing the configuration file: /config/core/app.conf

Clearing the configuration file: /config/registry/passwd

Clearing the configuration file: /config/registry/config.yml

Clearing the configuration file: /config/registry/root.crt

Clearing the configuration file: /config/registryctl/env

Clearing the configuration file: /config/registryctl/config.yml

Clearing the configuration file: /config/db/env

Clearing the configuration file: /config/jobservice/env

Clearing the configuration file: /config/jobservice/config.yml

Generated configuration file: /config/portal/nginx.conf

Generated configuration file: /config/log/logrotate.conf

Generated configuration file: /config/log/rsyslog_docker.conf

Generated configuration file: /config/nginx/nginx.conf

Generated configuration file: /config/core/env

Generated configuration file: /config/core/app.conf

Generated configuration file: /config/registry/config.yml

Generated configuration file: /config/registryctl/env

Generated configuration file: /config/registryctl/config.yml

Generated configuration file: /config/db/env

Generated configuration file: /config/jobservice/env

Generated configuration file: /config/jobservice/config.yml

loaded secret from file: /data/secret/keys/secretkey

Generated configuration file: /config/chartserver/env

Generated configuration file: /compose_location/docker-compose.yml

Clean up the input dir

Note: stopping existing Harbor instance ...

Stopping harbor-jobservice ... done

Stopping nginx             ... done

Stopping harbor-core       ... done

Stopping harbor-portal     ... done

Stopping redis             ... done

Stopping registryctl       ... done

Stopping registry          ... done

Stopping harbor-db         ... done

Stopping harbor-log        ... done

Removing harbor-jobservice ... done

Removing nginx             ... done

Removing harbor-core       ... done

Removing harbor-portal     ... done

Removing redis             ... done

Removing registryctl       ... done

Removing registry          ... done

Removing harbor-db         ... done

Removing harbor-log        ... done

Removing network harbor_harbor

Removing network harbor_harbor-chartmuseum

WARNING: Network harbor_harbor-chartmuseum not found.

[Step 5]: starting Harbor ...

Creating network "harbor_harbor" with the default driver

Creating network "harbor_harbor-chartmuseum" with the default driver

Creating harbor-log ... done

Creating harbor-db     ... done

Creating registry      ... done

Creating registryctl   ... done

Creating redis         ... done

Creating harbor-portal ... done

Creating chartmuseum   ... done

Creating harbor-core   ... done

Creating nginx             ... done

Creating harbor-jobservice ... done

✔ ----Harbor has been installed and started successfully.----

这之后，就可以用上述 harbor 来管理 helm charts。

3 图形界面操作

3.1 创建项目

首先，需要在 harbor 上创建一个名为 helm-repo 的项目，如图所示：

3.2 创建test用户

**-------------------------------kmaster服务器

相关操作如下----------------------------**

4 k8s集群一侧，安装helm等

4.1 安装helm

$ wget https://get.helm.sh/helm-v3.4.2-linux-amd64.tar.gz

$ tar -zxvf helm-v3.4.2-linux-amd64.tar.gz

$ cd linux-amd64/

# 拷贝helm到 /usr/local/bin

$ cp helm /usr/local/bin

4.2 验证helm

$ helm

The Kubernetes package manager

Common actions for Helm:

- helm search:    search for charts
- helm pull:      download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts

$ helm version

version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTre                                                                                                  eState:"clean", GoVersion:"go1.14.13"}

5 chart仓库

5.1 添加 hostname

$ vi /etc/hosts

192.168.8.131 reg.local.com

5.1 添加chart仓库

5.1.1 添加认证

cat /etc/docker/daemon.json

{"exec-opts":["native.cgroupdriver=systemd"],"log-driver":"json-file","log-opts":{"max-size":"100m"},"insecure-registries": ["reg.local.com"]}

**-------------------------------harbor服务的认证信息

拷贝到kmaster服务器----------------------------**

5.1.2 把harbor的认证信息拷贝到kmaster服务器的相同位置

[root@harbor cert]# ll

total 32

-rw-r--r--. 1 root root 2033 Dec 31 04:25 ca.crt

-rw-r--r--. 1 root root 3243 Dec 31 04:25 ca.key

-rw-r--r--. 1 root root   17 Dec 31 04:25 ca.srl

-rw-r--r--. 1 root root 2094 Dec 31 04:25 reg.local.com.cert

-rw-r--r--. 1 root root 2094 Dec 31 04:25 reg.local.com.crt

-rw-r--r--. 1 root root 1708 Dec 31 04:25 reg.local.com.csr

-rw-r--r--. 1 root root 3247 Dec 31 04:25 reg.local.com.key

-rw-r--r--. 1 root root  263 Dec 31 04:25 v3.ext

[root@kmaster cert]# ll

total 32

-rw-r--r-- 1 root root 2033 Dec 31 04:25 ca.crt

-rw-r--r-- 1 root root 3243 Dec 31 04:25 ca.key

-rw-r--r-- 1 root root   17 Dec 31 04:25 ca.srl

-rw-r--r-- 1 root root 2094 Dec 31 04:25 reg.local.com.cert

-rw-r--r-- 1 root root 2094 Dec 31 04:25 reg.local.com.crt

-rw-r--r-- 1 root root 1708 Dec 31 04:25 reg.local.com.csr

-rw-r--r-- 1 root root 3247 Dec 31 04:25 reg.local.com.key

-rw-r--r-- 1 root root  263 Dec 31 04:25 v3.ext

-------------------------------kmaster服务器相关操作如下----------------------------

5.1.3 追加私有仓库

$ helm repo add stable https://charts.helm.sh/stable

$ helm repo add --ca-file /data/cert/ca.crt --cert-file /data/cert/reg.local.com.cert --key-file /data/cert/reg.local.com.key test https://reg.local.com/chartrepo/helm-repo

"test" has been added to your repositories

5.2 更新chart列表

$ helm repo update

Hang tight while we grab the latest from your chart repositories...

...Successfully got an update from the "test" chart repository

...Successfully got an update from the "stable" chart repository

5.3 使用helm安装mysql

$ helm install stable/mysql --generate-name

5.4 查看mysql是否成功安装

$ helm ls

5.5 推送应用

helm3 现在默认不支持推送到charts库，需要安装插件helm-push

5.5.1 安装插件

$ helm plugin install https://github.com/chartmuseum/helm-push

Downloading and installing helm-push v0.9.0 ...

https://github.com/chartmuseum/helm-push/releases/download/v0.9.0/helm-push_0.9.0_linux_amd64.tar.gz

Installed plugin: push

5.5.2 查看追加的chart仓库

$ helm repo list

NAME    URL

stable  https://charts.helm.sh/stable

test    https://reg.local.com/chartrepo/helm-repo

5.5.3 创建chart

$ helm create mychart

Creating mychart

$ ll

total 8

drwxr-xr-x 2 root root    6 Jan  6 15:15 charts

-rw-r--r-- 1 root root 1098 Jan  6 15:15 Chart.yaml

drwxr-xr-x 3 root root  162 Jan  6 15:15 templates

-rw-r--r-- 1 root root 1800 Jan  6 15:15 values.yaml

5.5.4 检查chart

$ helm lint ./mychart

==> Linting ./mychart

Error unable to check Chart.yaml file in chart: stat mychart/Chart.yaml: no such file or directory

Error: 1 chart(s) linted, 1 chart(s) failed

[root@kmaster mychart]# cd ..

[root@kmaster ~]# helm lint ./mychart

==> Linting ./mychart

[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed

5.5.5 推送chart

$ helm push mychart test --ca-file /data/cert/ca.crt -u admin -p Harbor12345

Pushing mychart-0.1.0.tgz to test...

Done.

-------------------------------harbor服务器相关操作如下----------------------------

6 验证刚推送成功的chart

版本0.1.0

-------------------------------kmaster服务器相关操作如下----------------------------

7 从chart仓库安装

7.1 现有安装好的chart

$ helm list

NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION

7.2 安装mychart 0.1.0

$ helm install my1  mychart

NAME: my1

LAST DEPLOYED: Wed Jan  6 15:57:47 2021

NAMESPACE: default

STATUS: deployed

REVISION: 1

NOTES:

1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=mychart,app.kubernetes.io/instance=my1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT

7.3 卸载mychart 0.1.0

$ helm list

NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION

my1     default         1               2021-01-06 15:57:47.410255239 +0900 JST deployed        mychart-0.1.0   1.16.0

$ helm uninstall my1

release "my1" uninstalled

8 从Harbor拉web-demo镜像执行，看一下结果

8.1 镜像拉下来

docker pull reg.local.com/dev/web-demo:v1.0

v1.0: Pulling from dev/web-demo

7448db3b31eb: Pull complete

c36604fa7939: Pull complete

29e8ef0e3340: Pull complete

a0c934d2565d: Pull complete

a360a17c9cab: Pull complete

cfcc996af805: Pull complete

2cf014724202: Pull complete

4bc402a00dfe: Pull complete

7f506d2e6327: Pull complete

577ef7ca6761: Pull complete

Digest: sha256:80530996e66df715a51f3407423802a106271b1bc4267114d5a6cc5f9a0a87ef

Status: Downloaded newer image for reg.local.com/dev/web-demo:v1.0

reg.local.com/dev/web-demo:v1.0

8.2 查看一下镜像

$ docker images

REPOSITORY                           TAG           IMAGE ID       CREATED         SIZE

reg.local.com/dev/web-demo           v1.0          e03f3e0f181f   41 hours ago    677MB

8.3 执行镜像并把端口8080 外放成8888

docker run --publish 8888:8080 reg.local.com/dev/web-demo:v1.0

  .   ____          _            __ _ _

 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )

  '  |____| .__|_| |_|_| |_\__, | / / / /

 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::                (v2.4.1)

2021-01-06 17:41:12.950  INFO 1 --- [           main] com.local.demo.DemoApplication           : Starting DemoApplication v0.0.1-SNAPSHOT using Java 1.8.0_111 on 8f2fb5b3f985 with PID 1 (/app.jar started by root in /)

2021-01-06 17:41:12.957  INFO 1 --- [           main] com.local.demo.DemoApplication           : No active profile set, falling back to default profiles: default

2021-01-06 17:41:14.150  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)

2021-01-06 17:41:14.160  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]

2021-01-06 17:41:14.160  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]

2021-01-06 17:41:14.212  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext

2021-01-06 17:41:14.212  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1177 ms

2021-01-06 17:41:14.476  INFO 1 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'

2021-01-06 17:41:14.641  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''

2021-01-06 17:41:14.651  INFO 1 --- [           main] com.local.demo.DemoApplication           : Started DemoApplication in 2.136 seconds (JVM running for 2.486)

2021-01-06 17:41:30.169  INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'

2021-01-06 17:41:30.169  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'

2021-01-06 17:41:30.170  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms

8.4 浏览器查看一下画面

其他方式部署可以参考：k8s+jenkins实现自动化部署应用至k8s集群

原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。

如有侵权，请联系 cloudcommunity@tencent.com 删除。

编程算法

kubernetes

存储

原创声明：本文系作者授权腾讯云开发者社区发表，未经许可，不得转载。

如有侵权，请联系 cloudcommunity@tencent.com 删除。

编程算法

kubernetes

存储