jx是云原生CICD,devops的一个最佳实践之一,目前在快速的发展成熟中。最近调研了JX,这里为第3篇,介绍下如何安装jenkins x。
这里的服务器为centos 7。
使用官方的ceph-deploy安装即可,先安装ceph-deploy,然后每台机器安装运行环境
pip install ceph-deploy
export CEPH_DEPLOY_REPO_URL=http://mirrors.ustc.edu.cn/ceph/rpm-jewel/el7
export CEPH_DEPLOY_GPG_URL=http://mirrors.ustc.edu.cn/ceph/keys/release.asc
ceph-deploy install docker86-156 docker86-155 docker86-154
然后安装集群
ceph-deploy new docker86-156 docker86-155 docker86-154
修改配置文件
cat <<EOF >>ceph.conf
#osd_journal_size = 10000
public network = 192.168.86.0/24
osd_pool_default_size = 2
osd_pool_default_min_size = 1
osd_crush_chooseleaf_type = 1
osd_crush_update_on_start = true
max_open_files = 131072
osd pool default pg num = 128
osd pool default pgp num = 128
mon_pg_warn_max_per_osd = 0
mon clock drift allowed = 2
mon clock drift warn backoff = 30
mon_pg_warn_max_per_osd = 300
EOF
分发配置文件:
ceph-deploy --overwrite-conf config push docker86-155 docker86-154 docker86-156
安装服务
ceph-deploy mon create-initial
ceph-deploy admin docker86-156 docker86-155 docker86-154
安装osd
ceph-deploy disk zap docker86-156:sdb docker86-155:sdb docker86-154:sdb
ceph-deploy osd prepare docker86-156:sdb docker86-155:sdb docker86-154:sdb
ceph-deploy osd activate docker86-156:sdb1 docker86-154:sdb1
ceph osd pool create k8smeta 128
ceph osd pool create k8sdata 128
ceph fs new k8s k8smeta k8sdata
ceph osd pool ls detail
生成 Ceph secret
grep key /etc/ceph/ceph.client.admin.keyring |awk '{printf "%s", $NF}'|base64
假如得到: $SECRET==
在k8s创建Secret
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret
namespace: default
type: "kubernetes.io/rbd"
data:
key: $SECRET==
EOF
创建StorageClass
cat <<EOF | kubectl apply -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ceph-web
provisioner: kubernetes.io/rbd
parameters:
monitors: 192.168.86.156,192.168.86.155,192.168.86.154
adminId: admin
adminSecretName: ceph-secret
adminSecretNamespace: default
pool: rbd
userId: admin
userSecretName: ceph-secret
EOF
可以将ceph设为默认的storage class:
kubectl patch storageclass ceph-web -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
如果已经有git的,或者直接使用GitHub的跳过。
创建PV:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
name: cephfs-github-pv
namespace: gitea
labels:
name: cephfs-github-pv
spec:
capacity:
storage: 200Gi
accessModes:
- ReadWriteMany
cephfs:
monitors:
- 192.168.86.156:6789
path: /github
user: admin
secretRef:
name: ceph-secret
readOnly: false
persistentVolumeReclaimPolicy: Retain
EOF
PVC
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: cephfs-github-pvc
namespace: gitea
spec:
accessModes:
- ReadWriteMany
storageClassName: ""
resources:
requests:
storage: 200Gi
selector:
matchLabels:
name: cephfs-github-pv
EOF
gitea部署:
cat <<EOF | kubectl apply -f -
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: giteamysql
namespace: gitea
spec:
strategy:
type: Recreate
template:
metadata:
labels:
app: giteamysql
spec:
containers:
- image: gitea/gitea:latest
imagePullPolicy: IfNotPresent
name: gitea
resources: {}
volumeMounts:
- name: ceph
mountPath: /data
volumes:
- name: ceph
persistentVolumeClaim:
claimName: cephfs-github-pvc
EOF
创建服务:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
name: giteamysql-service
namespace: gitea
labels:
app: charts
spec:
ports:
- port: 80
targetPort: 3000
selector:
app: giteamysql
type: NodePort
EOF
创建ingress
cat <<EOF | kubectl apply -f -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "0"
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
kubernetes.io/tls-acme: 'true'
name: giteamysql-ingress
namespace: gitea
spec:
rules:
- host: github.youdomain.com
http:
paths:
- backend:
serviceName: giteamysql-service
servicePort: 80
path: /
EOF
一切正常的话,打开github.youdomain.com,按提示进行安装,设置管理员密码即可。
安装好后创建一个token, $git_access_token
将域名的通配符,a记录到k8s集群。
申请TLS证书,使用certbot
$ yum -y install yum-utils
$ yum-config-manager --enable rhui-REGION-rhel-server-extras rhui-REGION-rhel-server-optional
$ sudo yum install certbot
然后申请
certbot certonly --manual -d *.domain.com --email youmail@domain.com
这个会要求创建一个A记录,按提示创建即可。
一些OK的话,服务器/etc/letsencrypt/live/domain.com/ 会生成tls证书。
$ ll /etc/letsencrypt/live/iflyresearch.com/
total 4
lrwxrwxrwx. 1 root root 40 Oct 17 15:11 cert.pem -> ../../archive/iflyresearch.com/cert1.pem
drwxr-xr-x 2 root root 78 Nov 14 09:33 certs
lrwxrwxrwx. 1 root root 41 Oct 17 15:11 chain.pem -> ../../archive/iflyresearch.com/chain1.pem
lrwxrwxrwx. 1 root root 45 Oct 17 15:11 fullchain.pem -> ../../archive/iflyresearch.com/fullchain1.pem
lrwxrwxrwx. 1 root root 43 Oct 17 15:11 privkey.pem -> ../../archive/iflyresearch.com/privkey1.pem
要在k8s使用,需要创建secret:
kubectl create secret tls research-tls-secret --cert=cert.pem --key=./privkey.pem -n=kube-system
jx依赖helm,需要先安装,可以参见本系列第一篇
先创建一个namespace:incubation
写入 ceph-secret:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret
namespace: incubation
type: "kubernetes.io/rbd"
data:
key: $SECRET==
EOF
下载jx执行文件:
wget https://github.com/jenkins-x/jx/releases/download/v1.3.380/jx-linux-amd64.tar.gz
tar xzv jx-linux-amd64.tar.gz -C ~/.jx/bin
export PATH=$PATH:~/.jx/bin
echo 'export PATH=$PATH:~/.jx/bin' >> ~/.bashrc
然后执行安装命令:
jx install --external-ip=192.168.86.214 --namespace='incubation' --git-provider-url='http://github.iflyresearch.com' --git-username='jqpeng' --git-api-token=' $git_access_token' --domain='iflyresearch.com' --provider=kubernetes
然后按提示,输出jenkins等access_token等参数。
注意:
作者:Jadepeng 出处:jqpeng的技术记事本--http://www.cnblogs.com/xiaoqi 您的支持是对博主最大的鼓励,感谢您的认真阅读。 本文版权归作者所有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则保留追究法律责任的权利。