k8s自动化部署tendis/redis集群

原创

一个怀揣梦想的运维

发布于 2023-11-29 10:08:14

2830

发布于 2023-11-29 10:08:14

文章被收录于专栏：k8s自动化部署k8s自动化部署

自动化部署 tendis集群和redis集群初始化集群步骤一样，创建出3主3从的节点后，再创建一个初始化的job，对集群进行初始化配置。（redis集群创建修改镜像即可）

创建svc

以下是svc的yaml

kind: Service apiVersion: v1 metadata: name: tendis-cluster-auto-9n44 namespace: tendis labels: app: tendis-cluster-auto annotations: kubesphere.io/alias-name: tendis-cluster-auto kubesphere.io/serviceType: statefulservice spec: ports: - name: http-0 protocol: TCP port: 30032 targetPort: 51002 selector: app: tendis-cluster-auto clusterIP: None clusterIPs: - None type: ClusterIP sessionAffinity: None ipFamilies: - IPv4 ipFamilyPolicy: SingleStack internalTrafficPolicy: Cluster

创建6副本的statefulset

通过pvc模板自动创建对应的pvc挂载，可自行修改

kind: StatefulSet apiVersion: apps/v1 metadata: name: tendis-cluster-auto namespace: tendis labels: app: tendis-cluster-auto spec: replicas: 6 selector: matchLabels: app: tendis-cluster-auto template: metadata: creationTimestamp: null labels: app: tendis-cluster-auto annotations: kubesphere.io/imagepullsecrets: '{}' logging.kubesphere.io/logsidecar-config: '{}' spec: volumes: - name: host-time hostPath: path: /etc/localtime type: '' containers: - name: container-s140dj image: 'tencentdbforkv/tendisplus:latest' ports: - name: http-0 containerPort: 51002 protocol: TCP env: - name: CLUSTER value：yes - name: REDIS_PASSWORD value：test12345 resources: {} volumeMounts: - name: host-time mountPath: /etc/localtime - name: tendis-pvc mountPath: /data1/tendis/51002 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent restartPolicy: Always terminationGracePeriodSeconds: 30 dnsPolicy: ClusterFirst serviceAccountName: default serviceAccount: default securityContext: {} schedulerName: default-scheduler volumeClaimTemplates: - kind: PersistentVolumeClaim apiVersion: v1 metadata: name: tendis-pvc namespace: tendis creationTimestamp: null spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: default volumeMode: Filesystem status: phase: Pending serviceName: tendis-cluster-auto-9n44 podManagementPolicy: OrderedReady updateStrategy: type: RollingUpdate rollingUpdate: partition: 0 revisionHistoryLimit: 10

自制初始化镜像

docker pull tencentdbforkv/build:latest

这个镜像的系统是Centos8

在这个镜像的基础上更新yum源，然后安装dig命令，commit 提交成一个有dig命令的镜像。

修改build.sh 脚本

通过docker run -itd tencentdbforkv/build:latest sleep 36000s 然后进入容器

该脚本需传入4个环境变量参数

PODNAME，PASSWORD，NAMESPACE，SVCNAME,用于

拼接k8s的DNS域名解析地址${PODNAME}-$i.${SVCNAME}.${NAMESPACE}.svc.cluster.local

就能通过DNS域名去解析到各个节点去执行初始化操作。

然后替换里面的build.sh脚本为下面的脚本

#!/bin/bash set -ue sleep 10 PORT=51002 #PODNAME=$1 SIZE=3 #PASSWORD=$2 #NAMESPACE=$3 #SVCNAME=$4 slot_num=$((16384 / ${SIZE})) for (( i = 0; i < 6; i++)); do declare "ip$i"=”$(dig +short ${PODNAME}-$i.${SVCNAME}.${NAMESPACE}.svc.cluster.local)“ done for (( i = 0; i < 6; i++ )); do ip_var="ip$i" ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster meet ${!ip_var} ${PORT} done echo "cluster meet done......" sleep 10 ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster nodes for (( i = 0; i < 3; i++ )); do uuid[i]=$(./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster nodes | grep "ip$i" | awk '{print $1}') done for (( i = 3; i < 6; i++ )); do ./redis-cli -h "ip$i" -p ${PORT} -a ${PASSWORD} cluster replicate "${uuid[$((i-3))]}" done ./redis-cli -h ${ip0} -p ${PORT} -a ${PASSWORD} cluster addslots {0..5461} ./redis-cli -h ${ip1} -p ${PORT} -a ${PASSWORD} cluster addslots {5462..10922} ./redis-cli -h ${ip2} -p ${PORT} -a ${PASSWORD} cluster addslots {10923..16383} echo "cluster build done.." sleep 5 ./redis-cli -h ${IP}.10 -p ${PORT} -a ${PASSWORD} cluster nodes

保存脚本然后commit为最新的镜像

创建job初始化集群，并传入4个环境变量

kind: Job apiVersion: batch/v1 metadata: name: tendisbuild namespace: tendis labels: app: tendisbuild parallelism: 1 completions: 1 activeDeadlineSeconds: 60 backoffLimit: 2 selector: matchLabels: controller-uid: b2e57703-21fd-4011-8505-85353c6075f2 template: metadata: creationTimestamp: null labels: app: tendisbuild controller-uid: b2e57703-21fd-4011-8505-85353c6075f2 job-name: tendisbuild spec: containers: - name: container-j8xf02 image: 'harbor/build-tedis:v5' command: - /bin/bash - '-c' args: - cd /data1/build && sh build.sh ports: - name: http-0 containerPort: 20003 protocol: TCP env: - name: PASSWORD value: test12345 - name: DNS value: tendis-cluster-auto-9n44 - name: PODNAME value: tendis-cluster-auto - name: NAMESPACE value: tendis resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent restartPolicy: OnFailure terminationGracePeriodSeconds: 30 dnsPolicy: ClusterFirst serviceAccountName: default serviceAccount: default securityContext: {} schedulerName: default-scheduler completionMode: NonIndexed suspend: false

查看自动初始化后的集群信息