前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Helm部署Ingress使用DaemonSet+Taint/Tolerations+NodeSelector

Helm部署Ingress使用DaemonSet+Taint/Tolerations+NodeSelector

原创
作者头像
七辰
发布2023-10-01 10:01:40
3720
发布2023-10-01 10:01:40
举报
文章被收录于专栏:颜汐小屋颜汐小屋

kubernetes集群中需要在指定的几个节点上只部署Nginx Ingress Controller实例,不会跑其他业务容器。

环境说明

教程基于有k84集群,并安装好helm部署环境。强烈推荐使用helm发布您的代码!

执行helm version出现如下证明环境已经就绪。

  1. [root@oneinstack ~]# helm version
  2. Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
  3. Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

helm下载ingress chart

搜索ingress charts

  1. helm search ingress

下载nginx ingress

  1. helm fetch stable/nginx-ingress

fetch之后得到nginx-ingress-1.4.0.tgz

修改ingress helm chart

解压nginx-ingress-1.4.0.tgz

代码语言:javascript
复制
tar xzf nginx-ingress-1.4.0.tgz
注意: 解压过程出现implausibly old time stamp 1970-01-01 08:00:00可忽略
修改values.yaml,下面是我的修改好的:
## nginx configuration
## Ref: https://github.com/kubernetes/ingress/blob/master/controllers/nginx/configuration.md
##
controller:
  name: controller
  image:
    repository: quay.io/kubernetes-ingress-controller/nginx-ingress-controller  # 建议将镜像拖到自己私有参考,修改私有仓库地址
    tag: "0.24.1"
    pullPolicy: IfNotPresent
    # www-data -> uid 33
    runAsUser: 33
  config: {}
  # Will add custom header to Nginx https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/customization/custom-headers
  headers: {}
  # Required for use with CNI based kubernetes installations (such as ones set up by kubeadm),
  # since CNI and hostport don't mix yet. Can be deprecated once https://github.com/kubernetes/kubernetes/issues/23920
  # is merged
  hostNetwork: true   # 80 443 暴露到宿主机
  # Optionally change this to ClusterFirstWithHostNet in case you have 'hostNetwork: true'.
  # By default, while using host network, name resolution uses the host's DNS. If you wish nginx-controller
  # to keep resolving names inside the k8s network, use ClusterFirstWithHostNet.
  dnsPolicy: ClusterFirst
  ## Use host ports 80 and 443
  daemonset:
    useHostPort: false
    hostPorts:
      http: 80
      https: 443
      ## healthz endpoint
      stats: 18080
  ## Required only if defaultBackend.enabled = false
  ## Must be <namespace>/<service_name>
  ##
  defaultBackendService: ""
  ## Election ID to use for status update
  ##
  electionID: ingress-controller-leader
  ## Name of the ingress class to route through this controller
  ##
  ingressClass: nginx # 后续ingress.yaml annotations指定kubernetes.io/ingress.class: nginx
  # labels to add to the pod container metadata
  podLabels: {}
  #  key: value
  ## Allows customization of the external service
  ## the ingress will be bound to via DNS
  publishService:
    enabled: false
    ## Allows overriding of the publish service to bind to
    ## Must be <namespace>/<service_name>
    ##
    pathOverride: ""
  ## Limit the scope of the controller
  ##
  scope:
    enabled: false
    namespace: ""   # defaults to .Release.Namespace
  ## Additional command line arguments to pass to nginx-ingress-controller
  ## E.g. to specify the default SSL certificate you can use
  ## extraArgs:
  ##   default-ssl-certificate: "<namespace>/<secret_name>"
  extraArgs: {}
  ## Additional environment variables to set
  extraEnvs: []
  # extraEnvs:
  #   - name: FOO
  #     valueFrom:
  #       secretKeyRef:
  #         key: FOO
  #         name: secret-resource
  ## DaemonSet or Deployment
  ##
  kind: DaemonSet   #DaemonSet模式
  # The update strategy to apply to the Deployment or DaemonSet
  ##
  updateStrategy: {}
  #  rollingUpdate:
  #    maxUnavailable: 1
  #  type: RollingUpdate
  # minReadySeconds to avoid killing pods before we are ready
  ##
  minReadySeconds: 0
  ## Node tolerations for server scheduling to nodes with taints
  ## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
  ##
  tolerations:    #在节点上打污点,此处是容忍的key value effect
    - key: "nginx-ingress"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  affinity: {}
  ## Node labels for controller pod assignment
  ## Ref: https://kubernetes.io/docs/user-guide/node-selection/
  ##
  nodeSelector:
    nginx-ingress: "true"    #使用节点标签选择器,访问在所有节点运行ingress
  ## Liveness and readiness probe values
  ## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes
  ##
  livenessProbe:
    failureThreshold: 3
    initialDelaySeconds: 10
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 1
    port: 10254
  readinessProbe:
    failureThreshold: 3
    initialDelaySeconds: 10
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 1
    port: 10254
  ## Annotations to be added to controller pods
  ##
  podAnnotations:    # 支持prometheus抓取数据
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"
  replicaCount: 1
  minAvailable: 1
  resources: {}
  #  limits:
  #    cpu: 100m
  #    memory: 64Mi
  #  requests:
  #    cpu: 100m
  #    memory: 64Mi
  autoscaling:
    enabled: false
    minReplicas: 1
    maxReplicas: 11
    targetCPUUtilizationPercentage: 50
    targetMemoryUtilizationPercentage: 50
  ## Override NGINX template
  customTemplate:
    configMapName: ""
    configMapKey: ""
  service:
    annotations: {}
    labels: {}
    clusterIP: ""
    ## List of IP addresses at which the controller services are available
    ## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
    ##
    externalIPs: []
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    enableHttp: true
    enableHttps: true
    ## Set external traffic policy to: "Local" to preserve source IP on
    ## providers supporting it
    ## Ref: https://kubernetes.io/docs/tutorials/services/source-ip/#source-ip-for-services-with-typeloadbalancer
    externalTrafficPolicy: ""
    healthCheckNodePort: 0
    targetPorts:
      http: http
      https: https
    type: ClusterIP
    # type: NodePort
    # nodePorts:
    #   http: 32080
    #   https: 32443
    nodePorts:
      http: ""
      https: ""
  extraContainers: []
  ## Additional containers to be added to the controller pod.
  ## See https://github.com/lemonldap-ng-controller/lemonldap-ng-controller as example.
  #  - name: my-sidecar
  #    image: nginx:latest
  #  - name: lemonldap-ng-controller
  #    image: lemonldapng/lemonldap-ng-controller:0.2.0
  #    args:
  #      - /lemonldap-ng-controller
  #      - --alsologtostderr
  #      - --configmap=$(POD_NAMESPACE)/lemonldap-ng-configuration
  #    env:
  #      - name: POD_NAME
  #        valueFrom:
  #          fieldRef:
  #            fieldPath: metadata.name
  #      - name: POD_NAMESPACE
  #        valueFrom:
  #          fieldRef:
  #            fieldPath: metadata.namespace
  #    volumeMounts:
  #    - name: copy-portal-skins
  #      mountPath: /srv/var/lib/lemonldap-ng/portal/skins
  extraVolumeMounts: []
  ## Additional volumeMounts to the controller main container.
  #  - name: copy-portal-skins
  #   mountPath: /var/lib/lemonldap-ng/portal/skins
  extraVolumes: []
  ## Additional volumes to the controller pod.
  #  - name: copy-portal-skins
  #    emptyDir: {}
  extraInitContainers: []
  ## Containers, which are run before the app containers are started.
  # - name: init-myservice
  #   image: busybox
  #   command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']
  stats:
    enabled: true
    service:
      annotations: {}
      clusterIP: ""
      ## List of IP addresses at which the stats service is available
      ## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
      ##
      externalIPs: []
      loadBalancerIP: ""
      loadBalancerSourceRanges: []
      servicePort: 18080
      type: ClusterIP
  ## If controller.stats.enabled = true and controller.metrics.enabled = true, Prometheus metrics will be exported
  ##
  metrics:
    enabled: true
    service:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "10254"
      clusterIP: ""
      ## List of IP addresses at which the stats-exporter service is available
      ## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
      ##
      externalIPs: []
      loadBalancerIP: ""
      loadBalancerSourceRanges: []
      servicePort: 9913
      type: ClusterIP
    serviceMonitor:
      enabled: false
      additionalLabels: {}
      namespace: ""
  lifecycle: {}
  priorityClassName: ""
## Rollback limit
##
revisionHistoryLimit: 10
## Default 404 backend
##
defaultBackend:
  ## If false, controller.defaultBackendService must be provided
  ##
  enabled: true
  name: default-backend
  image:
    repository: registry.cn-hangzhou.aliyuncs.com/google_containers/defaultbackend
    tag: "1.4"
    pullPolicy: IfNotPresent
  extraArgs: {}
  port: 8080
  ## Node tolerations for server scheduling to nodes with taints
  ## Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
  ##
  tolerations: []
  #  - key: "key"
  #    operator: "Equal|Exists"
  #    value: "value"
  #    effect: "NoSchedule|PreferNoSchedule|NoExecute(1.6 only)"
  affinity: {}
  # labels to add to the pod container metadata
  podLabels: {}
  #  key: value
  ## Node labels for default backend pod assignment
  ## Ref: https://kubernetes.io/docs/user-guide/node-selection/
  ##
  nodeSelector: {}
  ## Annotations to be added to default backend pods
  ##
  podAnnotations: {}
  replicaCount: 1
  minAvailable: 1
  resources: {}
  # limits:
  #   cpu: 10m
  #   memory: 20Mi
  # requests:
  #   cpu: 10m
  #   memory: 20Mi
  service:
    annotations: {}
    clusterIP: ""
    ## List of IP addresses at which the default backend service is available
    ## Ref: https://kubernetes.io/docs/user-guide/services/#external-ips
    ##
    externalIPs: []
    loadBalancerIP: ""
    loadBalancerSourceRanges: []
    servicePort: 80
    type: ClusterIP
  priorityClassName: ""
## Enable RBAC as per https://github.com/kubernetes/ingress/tree/master/examples/rbac/nginx and https://github.com/kubernetes/ingress/issues/266
rbac:
  create: true
# If true, create & use Pod Security Policy resources
# https://kubernetes.io/docs/concepts/policy/pod-security-policy/
podSecurityPolicy:
  enabled: false
serviceAccount:
  create: true
  name:
## Optional array of imagePullSecrets containing private registry credentials
## Ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
imagePullSecrets: []
# - name: secretName
# TCP service key:value pairs
# Ref: https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/tcp
##
tcp: {}
#  8080: "default/example-tcp-svc:9000"
# UDP service key:value pairs
# Ref: https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx/examples/udp
##
udp: {}
#  53: "kube-system/kube-dns:53"

NODE打标签,设置污点

打标签

  1. kubectl label nodes 10.0.10.7 nginx-ingress=true
  2. kubectl label nodes 10.0.10.8 nginx-ingress=true
  3. kubectl label nodes 10.0.10.9 nginx-ingress=true

设置污点

  1. kubectl taint nodes 10.0.10.7 nginx-ingress=true:NoSchedule
  2. kubectl taint nodes 10.0.10.8 nginx-ingress=true:NoSchedule
  3. kubectl taint nodes 10.0.10.9 nginx-ingress=true:NoSchedule

安装nginx-ingress

  1. helm upgrade nginx-ingress ./nginx-ingress --install --namespace nginx-ingress --dry-run  # 测试运行
  2. helm upgrade nginx-ingress ./nginx-ingress --install --namespace nginx-ingress

注意:

  1. 需要先在NODE节点打污点、标签
  2. helm名字和命名空间请使用nginx-ingress, 和直接用yaml文件(ingress-nginx)有区别。否则DaemonSet、pod名字比较奇怪

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
作者已关闭评论
0 条评论
热度
最新
推荐阅读
目录
  • 环境说明
  • helm下载ingress chart
  • 修改ingress helm chart
  • NODE打标签,设置污点
  • 安装nginx-ingress
相关产品与服务
专用宿主机
专用宿主机(CVM Dedicated Host,CDH)提供用户独享的物理服务器资源,满足您资源独享、资源物理隔离、安全、合规需求。专用宿主机搭载了腾讯云虚拟化系统,购买之后,您可在其上灵活创建、管理多个自定义规格的云服务器实例,自主规划物理资源的使用。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档