前往小程序,Get更优阅读体验!
立即前往
首页
学习
活动
专区
工具
TVP
发布
社区首页 >专栏 >Ingress-Nginx 服务暴露基础学习与实践 (2)

Ingress-Nginx 服务暴露基础学习与实践 (2)

原创
作者头像
全栈工程师修炼指南
修改2021-07-26 11:17:17
3.2K0
修改2021-07-26 11:17:17
举报
文章被收录于专栏:全栈工程师修炼之路

如下基础示例请看上一篇文章

示例1.Ingress 常规使用方案

示例2.Ingress HTTPS 代理访问

示例3.Ingress Rewrite 重写重定向访问

示例4.Ingress VirtualHost 虚拟主机访问

示例5.Ingress BasicAuth(基础认证)功能

示例6.Ingress Redirect 域名重定向功能

示例7.Ingress 黑白名单访问限制

更多学习笔记文章请关注 WeiyiGeek 公众账号,学习交流【邮箱联系: Master#weiyigeek.top】

原文地址: https://mp.weixin.qq.com/s/ZxyNCP32BQjQo5ygztJttQ


示例8.Ingress Header 请求头匹配

描述: Ingress-nginx 的 匹配请求头,可以采用nginx.ingress.kubernetes.io/server-snippet注解。

代码语言:txt
复制
# - 资源清单

tee ingress-header-request.yaml <<'EOF'

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-heade

  namespace: demo

  annotations:

    nginx.ingress.kubernetes.io/server-snippet: |-

      set $agentflag 0;

      if ($http\_user\_agent ~\* "(Firefox)" ){

        set $agentflag 1;

      }

      if ($agentflag = 1) {

        return 301 https://www.weiyigeek.top;

      }

spec:

  rules: 

    - host: header.weiyigeek.top

      http:

        paths:

        - path: /

          pathType: ImplementationSpecific

          backend:

            service:

              name: backend-svc

              port:

                number: 8080

status:

  loadBalancer: {}

EOF



# - 创建查看 Ingress 资源对象

kubectl create -f ingress-header-request.yaml

kubectl get ing -n demo ingress-header && kubectl describe ing -n demo ingress-heade

  # Rules:

  #   Host                  Path  Backends

  #   ----                  ----  --------

  #   header.weiyigeek.top  /     backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)



# - 验证 Ingress 资源对象

# (1) 指定 User-Agent 请求头 包含 Firefox

$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' -k -i

  # HTTP/1.1 301 Moved Permanently

  # Date: Fri, 16 Jul 2021 14:00:04 GMT

  # Content-Type: text/html

  # Content-Length: 162

  # Connection: keep-alive

  # Location: https://www.weiyigeek.top



  # <html>

  # <head><title>301 Moved Permanently</title></head>

  # <body>

  # <center><h1>301 Moved Permanently</h1></center>

  # <hr><center>nginx</center>

  # </body>

  # </html>



# (2) 指定 User-Agent 请求头 不包含 Firefox 

$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0' -k -i

  # HTTP/1.1 200

  # Date: Fri, 16 Jul 2021 14:03:29 GMT

  # Content-Type: text/html

  # Content-Length: 40

  # Connection: keep-alive

  # Accept-Ranges: bytes

  # ETag: W/"40-1626351892000"

  # Last-Modified: Thu, 15 Jul 2021 12:24:52 GMT



  # hostname-web-backend-0-WeiyiGeek-Tomcat

示例9.Ingress 灰度金丝雀发布

描述: 在某些情况下,您可能希望通过向与生产服务不同的服务发送少量请求来“金丝雀”一组新的更改。金丝雀注解使 Ingress 规范能够根据应用的规则充当路由请求的替代服务,在nginx.ingress.kubernetes.io/canary: "true"设置后可以启用以下用于配置金丝雀的注释:。

Nginx Annotations 支持以下 4 种 Canary 规则:

代码语言:txt
复制
\* nginx.ingress.kubernetes.io/canary-weight:基于服务权重的流量切分,适用于蓝绿部署,权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为 100 意味着所有请求都将被发送到 Canary 入口。

\* nginx.ingress.kubernetes.io/canary-by-header:基于 Request Header 的流量切分,适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时,请求将会被一直发送到 Canary 版本;当 Request Header 设置为 never时,请求不会被发送到 Canary 入口;对于任何其他 Header 值,将忽略 Header,并通过优先级将请求与其他金丝雀规则进行优先级的比较。

\* nginx.ingress.kubernetes.io/canary-by-header-value:要匹配的 Request Header 的值,用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时,它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值,必须与上一个 annotation (即:canary-by-header)一起使用。

\* nginx.ingress.kubernetes.io/canary-by-header-pattern:这与canary-by-header-valuePCRE 正则表达式匹配的工作方式相同。请注意,canary-by-header-value设置此注释时将被忽略。当给定的 Regex 在请求处理过程中导致错误时,该请求将被视为不匹配。

\* nginx.ingress.kubernetes.io/canary-by-cookie:基于 Cookie 的流量切分,适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时,它将被路由到 Canary 入口;当 cookie 值设置为 never时,请求不会被发送到 Canary 入口;对于任何其他值,将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。 定义两个版本的代码。

Tips : 金丝雀规则是按优先顺序计算的。优先级如下:canary by header->canary by cookie->canary weight

Tips : 注意当您将入口标记为 Canary 时,除nginx.ingress.kubernetes.io/load-balanceand之外的所有其他非 Canary 注释都将被忽略(从相应的主入口继承)nginx.ingress.kubernetes.io/upstream-hash-by。`

**资源配置清单:**

代码语言:txt
复制
# - 默认访问前端

tee ingress-canary-v1.yaml <<'EOF'

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-canary-v1

  namespace: demo

  annotations:

    nginx.ingress.kubernetes.io/canary: "true"

    nginx.ingress.kubernetes.io/canary-by-header: canary

    nginx.ingress.kubernetes.io/canary-by-header-value: v1

    nginx.ingress.kubernetes.io/canary-weight: "50"

spec:

  rules: 

    - host: canary.weiyigeek.top

      http:

        paths:

        - path: /

          pathType: ImplementationSpecific

          backend:

            service:

              name: front-svc

              port:

                number: 80

EOF



# - 利用金丝雀访问后端

tee ingress-canary-v2.yaml <<'EOF'

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-canary-v2

  namespace: demo

  annotations:

    nginx.ingress.kubernetes.io/canary: "true"

    nginx.ingress.kubernetes.io/canary-by-header: canary

    nginx.ingress.kubernetes.io/canary-by-header-value: v2

    nginx.ingress.kubernetes.io/canary-weight: "50"

spec:

  rules: 

    - host: canary.weiyigeek.top

      http:

        paths:

        - path: /

          pathType: ImplementationSpecific

          backend:

            service:

              name: backend-svc

              port:

                number: 8080

EOF



~/k8s/ingress/demo1# kubectl create -f ingress-canary-v1.yaml

  # ingress.networking.k8s.io/ingress-canary-v1 created

~/k8s/ingress/demo1# kubectl create -f ingress-canary-v2.yaml

  # ingress.networking.k8s.io/ingress-canary-v2 created



~/k8s/ingress/demo1# kubectl get ing -n demo ingress-canary-v1 ingress-canary-v2

  # NAME                CLASS    HOSTS                  ADDRESS   PORTS   AGE

  # ingress-canary-v1   <none>   canary.weiyigeek.top             80      3m12s

  # ingress-canary-v2   <none>   canary.weiyigeek.top             80      3m4s



# 验证金丝雀 Ingress 对象应用

for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -i;done

for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -H 'weiyigeek: v2' -i;done

示例10.Ingress 相关配置补充

  • **(1) 自定义错误页面**

参考地址: https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/customization/custom-errors/custom-default-backend.yaml

描述: 可以修改 deploy.yaml 中指定 nginx-ingress-controller 运行的参数,在此处添加- --default-backend-service=名称空间/SVC名称参数

代码语言:txt
复制
$ grep -A 3 -n "/nginx-ingress" deploy.yaml

332:            - /nginx-ingress-controlle

333-            - --publish-service=$(POD\_NAMESPACE)/ingress-nginx-controlle

334-            - --election-id=ingress-controller-leade

335-            - --ingress-class=nginx

                - --default-backend-service=demo/error-svc
  • **(2) 连接和传输速率限制**

描述: 下述注释定义了对连接和传输速率的限制,这些可以用来减轻DDoS攻击。

代码语言:txt
复制
\* nginx.ingress.kubernetes.io/limit-connections:单个IP地址允许的并发连接数。超出此限制时,将返回503错误。

\* nginx.ingress.kubernetes.io/limit-rps:每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。

\* nginx.ingress.kubernetes.io/limit-rpm:每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。

\* nginx.ingress.kubernetes.io/limit-burst-multiplier:突发大小限制速率的倍数。默认的脉冲串乘数为5,此注释将覆盖默认的乘数。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。

\* nginx.ingress.kubernetes.io/limit-rate-after:最初的千字节数,在此之后,对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。

\* nginx.ingress.kubernetes.io/limit-rate:每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。

\* nginx.ingress.kubernetes.io/limit-whitelist:客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。

**示例说明:**

代码语言:txt
复制
# - 资源清单配置:

tee ingress-limit.yaml <<'EOF'

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-limit

  namespace: demo

  annotations:

    nginx.ingress.kubernetes.io/limit-rate: "100K"               # 限制客户端每秒传输的字节数

    nginx.ingress.kubernetes.io/limit-whitelist: "10.10.107.220" # 白名单中的IP不限速

    nginx.ingress.kubernetes.io/limit-rps: 1                     # 单个IP每秒的连接数

    nginx.ingress.kubernetes.io/limit-rpm: 30                    # 单个IP每分钟的连接数

spec:

  rules: 

    - host: limit.weiyigeek.top

      http:

        paths:

        - path: /

          pathType: ImplementationSpecific

          backend:

            service:

              name: backend-svc

              port:

                number: 8080

status:

  loadBalancer: {}

EOF
  • **(3) ConfigMap在Ingress Controller中实战**

描述: 前面我们说到我们除了可以采用annotations方式配置ingress规则外也可以通过ConfigMap来存储相应的Ingress规则;

Tips : 通过 Ingress ConfigMaps 查看详细的配置参数。

Ingress 规则存储配置清单:

代码语言:txt
复制
# --configmap=

tee nginx-config.yaml <<'EOF'

kind: ConfigMap

apiVersion: v1

metadata:

  name: ingress-nginx-controller  # 注意此ConfigMap名称与/nginx-ingress-controller 中 `--configmap=$(POD\_NAMESPACE)/ingress-nginx-controller` 对应

  namespace: ingress-nginx

  labels:

    app: ingress-nginx

data:

  proxy-connect-timeout: "15"   #这里有个坑,到家一定要注意是'-'不是'\_'

EOF



# --tcp-services-configmap=

---

kind: ConfigMap

apiVersion: v1

metadata:

  name: tcp-services

  namespace: ingress-nginx

  labels:

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/part-of: ingress-nginx



# --udp-services-configmap

---

kind: ConfigMap

apiVersion: v1

metadata:

  name: udp-services

  namespace: ingress-nginx

  labels:

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/part-of: ingress-nginx

nginx-ingress-controller 使用 Configmap 作为配置存在控制器:

代码语言:txt
复制
# 可以在 nginx-ingress-controller 部署资源清单中看见`--configmap=`参数

vim +328 deploy.yaml

 containers:

        - name: controlle

          image: pollyduan/ingress-nginx-controller:v0.47.0

          imagePullPolicy: IfNotPresent

          lifecycle:

            preStop:

              exec:

                command:

                  - /wait-shutdown

          args:

            - /nginx-ingress-controlle

            - --publish-service=$(POD\_NAMESPACE)/ingress-nginx-controlle

            - --election-id=ingress-controller-leade

            - --ingress-class=nginx

            - --configmap=$(POD\_NAMESPACE)/ingress-nginx-controller   # 使用 confimap 配置文件

            - --validating-webhook=:8443

            - --validating-webhook-certificate=/usr/local/certificates/cert

            - --validating-webhook-key=/usr/local/certificates/key

查看ingress-nginx名称空间中的ConfigMap配置

代码语言:txt
复制
kubectl apply -f nginx-config.yaml

  # configmap/ingress-nginx-controller configured



kubectl get cm -n ingress-nginx

  # NAME                              DATA   AGE

  # ingress-controller-leader-nginx   0      2d8h

  # ingress-nginx-controller          0      2d8h

  # kube-root-ca.crt                  1      2d8h



kubectl describe cm -n ingress-nginx ingress-nginx-controlle

  # Name:         ingress-nginx-controlle

  # Namespace:    ingress-nginx

  # Labels:       app.kubernetes.io/component=controlle

  #               app.kubernetes.io/instance=ingress-nginx

  #               app.kubernetes.io/managed-by=Helm

  #               app.kubernetes.io/name=ingress-nginx

  #               app.kubernetes.io/version=0.47.0

  #               helm.sh/chart=ingress-nginx-3.33.0

  # Annotations:  <none>



  # Data

  # ====

  # proxy-connect-timeout:

  # ----

  # 15

  # Events:

  #   Type    Reason  Age   From                      Message

  #   ----    ------  ----  ----                      -------

  #   Normal  UPDATE  25s   nginx-ingress-controller  ConfigMap ingress-nginx/ingress-nginx-controlle

验证是否在ingress-nginx-controller中配置生效:

代码语言:txt
复制
kubectl get pod -n ingress-nginx -l app.kubernetes.io/component=controlle

  # NAME                                        READY   STATUS    RESTARTS   AGE

  # ingress-nginx-controller-5df4f79b95-6p8rr   1/1     Running   0          2d8h



# 在 ingress-nginx-controller-5df4f79b95-6p8r 的 Pod 中 查看 nginx.conf 文件中 proxy\_connect\_timeout 参数是否为 15s

root@k8s-master-1:~/k8s/ingress/deployment# kubectl -n ingress-nginx exec -it ingress-nginx-controller-5df4f79b95-6p8rr bash

bash-5.1$ grep "proxy\_connect\_timeout" nginx.conf

          proxy\_connect\_timeout                   15s;   # 

企业实战

1.利用Ingress访问后端的Nginx Web和Tomcat后端综合实践。

**实验目标:**

  • 1) 在腾讯云创建test.weiyigeek.top以及demo.weiyigeek.top证书以供Ingress使用。
  • 2) 分别在kubernetes集群中创建Nginx WebTomcat 后端应用。
  • 3) 分别使用Ingress控制器创建以SSL访问/为NginxWeb而/tomcatTomcat 后端需要BasicAuth(基础认证)访问功能

**详细流程:**

WeiyiGeek.SSL证书申请
WeiyiGeek.SSL证书申请
  • Step 2.分别在kubernetes集群中创建Nginx WebTomcat 后端应用以及创建名称空间。
代码语言:txt
复制
tee sts-svc-nginx-tomcat.yaml<<'EOF'

apiVersion: v1

kind: Namespace

metadata:

  name: demo

---

# Service 服务创建(服务是软件服务的命名抽象)。

# kubectl api-resources | grep "service"

apiVersion: v1

kind: Service

metadata:

  name: front-svc

  namespace: demo

  labels:

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/part-of: ingress-nginx

spec:

  selector:

    app: nginx-web

  ports:

    - port: 80 

      targetPort: 80 

      protocol: TCP 

---

apiVersion: v1

kind: Service

metadata:

  name: backend-svc

  namespace: demo

  labels:

    app.kubernetes.io/name: ingress-nginx

    app.kubernetes.io/part-of: ingress-nginx

spec:

  selector:

    app: tomcat-web

  ports:

    - port: 8080

      targetPort: 8080 

      protocol: TCP

  sessionAffinity: None

---

apiVersion: apps/v1

kind: StatefulSet

metadata:

  name: web-front

  namespace: demo

spec:

  replicas: 2

  serviceName: front-svc

  selector:

    matchLabels:

      app: nginx-web

  template:

    metadata:

      labels:

        app: nginx-web

    spec:

      initContainers:

      - name: init-html

        image: busybox:latest

        imagePullPolicy: IfNotPresent

        command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/share/nginx/html/index.html"]

        env:

        - name: MSG

          value: "WeiyiGeek-Nginx"

        volumeMounts:

        - name: web

          mountPath: "/usr/share/nginx/html"

        securityContext:

          privileged: true

      containers:

      - name: nginx   

        image: nginx:latest

        imagePullPolicy: IfNotPresent

        env:

        - name: TZ

          value: "Asia/Shanghai"

        ports:

        - name: nginx

          containerPort: 80

        volumeMounts:

        - name: web

          mountPath: "/usr/share/nginx/html"

      volumes:

      - name: web 

        emptyDir: {}

---

apiVersion: apps/v1

kind: StatefulSet

metadata:

  name: web-backend

  namespace: demo

spec:

  replicas: 2

  serviceName: backend-svc

  selector:

    matchLabels:

      app: tomcat-web

  template:

    metadata:

      labels:

        app: tomcat-web

    spec:

      initContainers:

      - name: init-html

        image: busybox:latest

        imagePullPolicy: IfNotPresent

        command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/local/tomcat/webapps/ROOT/index.html"]

        env:

        - name: MSG

          value: WeiyiGeek-Tomcat

        volumeMounts:

        - name: web

          mountPath: "/usr/local/tomcat/webapps/ROOT/"

        securityContext:

          privileged: true

      containers:

      - name: nginx   

        image: "tomcat:8.5.69-jdk8-openjdk-slim-buster"

        imagePullPolicy: IfNotPresent

        env:

        - name: TZ

          value: "Asia/Shanghai"

        ports:

        - name: nginx

          containerPort: 8080

        volumeMounts:

        - name: web

          mountPath: "/usr/local/tomcat/webapps/ROOT/"

      volumes:

      - name: web 

        emptyDir: {}

EOF



# 利用资源清单进行创建nginx、Tomcat应用

kubectl create -f sts-svc-nginx-tomcat.yaml

  # namespace/demo created

  # service/front-svc created

  # service/backend-svc created

  # statefulset.apps/web-front created

  # statefulset.apps/web-backend created
  • Step 3.验证SVC服务负载均衡和请求查看创建的应用
代码语言:txt
复制
# 查看 Service 控制器对象

$ kubectl get svc -n demo -o wide

  # NAME          TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE   SELECTOR

  # backend-svc   ClusterIP   None         <none>        8080/TCP   12m   name=tomcat-web

  # front-svc     ClusterIP   None         <none>        80/TCP     12m   name=nginx-web



# 查看 StatefulSet 控制器对象

$ kubectl get sts -n demo -o wide

  # NAME          READY   AGE   CONTAINERS   IMAGES

  # web-backend   2/2     13m   nginx        tomcat:8.5.69-jdk8-openjdk-slim-buste

  # web-front     2/2     13m   nginx        nginx:latest



# 查看 nginx 与 tomcat 相关 Pod 信息

$ kubectl get pod -n demo -o wide --show-labels

  # NAME            READY   STATUS    RESTARTS   AGE   IP               NODE        LABELS

  # web-backend-0   1/1     Running   0          13m   192.168.109.90   k8s-node-1  app=tomcat-web,controller-revision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-0

  # web-backend-1   1/1     Running   0          13m   192.168.109.93   k8s-node-1  app=tomcat-web,controller-revision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-1

  # web-front-0     1/1     Running   0          13m   192.168.109.91   k8s-node-1  app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-0

  # web-front-1     1/1     Running   0          13m   192.168.109.92   k8s-node-1  app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-1



# 查看 SVC 服务发现利用CoreDNS进行域名解析

$ nslookup front-svc.demo.svc.cluster.local. 10.96.0.10

  # Server:         10.96.0.10

  # Address:        10.96.0.10#53

  # Name:   front-svc.demo.svc.cluster.local

  # Address: 192.168.109.91

  # Name:   front-svc.demo.svc.cluster.local

  # Address: 192.168.109.92

$ nslookup backend-svc.demo.svc.cluster.local. 10.96.0.10

  # ...

  # Name:   backend-svc.demo.svc.cluster.local

  # Address: 192.168.109.90

  # Name:   backend-svc.demo.svc.cluster.local



# 请求查看 Nginx 与 Tomcat 应用回显。

curl front-svc.demo.svc.cluster.local

  # hostname-web-front-0-WeiyiGeek-Nginx

curl front-svc.demo.svc.cluster.local

  # hostname-web-front-1-WeiyiGeek-Nginx

curl backend-svc.demo.svc.cluster.local:8080

  # hostname-web-backend-1-WeiyiGeek-Tomcat

curl backend-svc.demo.svc.cluster.local:8080

  # hostname-web-backend-0-WeiyiGeek-Tomcat
代码语言:txt
复制
# Tips:  kubectl api-resources  | grep "secret" 可查看apiversions以及namespace是否支持

# 假设你已经将证书上传到k8s集群目录中并解压

$ ls

demo.weiyigeek.top.zip  test.weiyigeek.top.zip



$ unzip demo.weiyigeek.top.zip -d demo.weiyigeek.top

$ unzip test.weiyigeek.top.zip -d test.weiyigeek.top



# 创建 tls 类型的 secret

$ cd /tmp/demo.weiyigeek.top/Nginx

kubectl -n demo create secret tls demoweiyigeektop --key 2\_demo.weiyigeek.top.key --cert 1\_demo.weiyigeek.top\_bundle.crt

  # secret/demoweiyigeektop created



$ cd /tmp/test.weiyigeek.top/Nginx

kubectl -n demo create secret tls testweiyigeektop --key 2\_test.weiyigeek.top.key --cert 1\_test.weiyigeek.top\_bundle.crt

  # secret/testweiyigeektop created
  • Step 4.创建Ingress控制器的配置清单文件及Ingress对应规则。
代码语言:txt
复制
tee ingress-demo.yaml <<'EOF'

apiVersion: networking.k8s.io/v1

kind: Ingress

metadata:

  name: ingress-demo

  namespace: demo

  annotations:

    # 指定 Ingress Controller 的类型

    kubernetes.io/ingress.class: "nginx"

    # 指定我们的 rules 的 path 可以使用正则表达式

    nginx.ingress.kubernetes.io/use-regex: "true"

    # 连接超时时间,默认为 5s

    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"

    # 后端服务器回转数据超时时间,默认为 60s

    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"

    # 后端服务器响应超时时间,默认为 60s

    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"

    # 客户端上传文件,最大大小,默认为 20m

    nginx.ingress.kubernetes.io/proxy-body-size: "10m"

    # URL 重写

    nginx.ingress.kubernetes.io/rewrite-target: /

spec:

  # 默认后端服务指定

  defaultBackend:

    service:

      name: front-svc

      port:

        number: 80

  tls:

  - hosts:

    - test.weiyigeek.top

    secretName: testweiyigeektop

  - hosts:

    - demo.weiyigeek.top

    secretName: demoweiyigeektop

  rules:

  - host: test.weiyigeek.top

    http:

      paths:

      - pathType: Prefix

        path: /tomcat

        backend:

          service:

            name: backend-svc

            port:

              number: 8080

      - pathType: Prefix

        path: /nginx

        backend:

          service:

            name: front-svc

            port:

              number: 80

  - host: demo.weiyigeek.top

    http:

      paths:

      - pathType: Prefix

        path: /

        backend:

          service:

            name: front-svc

            port:

              number: 80

EOF
  • Step 5.创建Ingress并查看相关信息
代码语言:txt
复制
$ kubectl create -f ingress-demo.yaml

  # ingress.networking.k8s.io/ingress-demo created  



# 查看 ingress-nginx 的 SVC 

kubectl get svc -n ingress-nginx

  # NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE

  # ingress-nginx-controller             LoadBalancer   10.104.171.212   <pending>     80:31122/TCP,443:32155/TCP   24h

  # ingress-nginx-controller-admission   ClusterIP      10.99.168.221    <none>        443/TCP                      24h



# 查看 ingress-demo 对象相关信息

$ kubectl get ing -n demo ingress-demo

  # NAME           CLASS    HOSTS                                   ADDRESS   PORTS     AGE

  # ingress-demo   <none>   test.weiyigeek.top,demo.weiyigeek.top             80, 443   24h



$ kubectl describe ing -n demo ingress-demo

  # Name:             ingress-demo

  # Namespace:        demo

  # Address:

  # Default backend:  front-svc:80 (192.168.109.91:80,192.168.109.92:80)

  # TLS:

  #   testweiyigeektop terminates test.weiyigeek.top

  #   demoweiyigeektop terminates demo.weiyigeek.top

  # Rules:

  #   Host                Path  Backends

  #   ----                ----  --------

  #   test.weiyigeek.top /tomcat   backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)

  #   demo.weiyigeek.top /         front-svc:80 (192.168.109.91:80,192.168.109.92:80)

  # Annotations:          kubernetes.io/ingress.class: nginx

  #                       nginx.ingress.kubernetes.io/proxy-body-size: 10m

  #                       nginx.ingress.kubernetes.io/proxy-connect-timeout: 600

  #                       nginx.ingress.kubernetes.io/proxy-read-timeout: 600

  #                       nginx.ingress.kubernetes.io/proxy-send-timeout: 600

  #                       nginx.ingress.kubernetes.io/rewrite-target: /

  #                       nginx.ingress.kubernetes.io/use-regex: true

  # Events:               <none>
  • Step 6.验证Ingress规则配置的应用分别访问 https://demo.weiyigeek.tophttps://test.weiyigeek.top 站点。
代码语言:txt
复制
$ curl 'https://10.10.107.221' -H 'host: demo.weiyigeek.top' -i -k

  # HTTP/2 200

  # date: Sat, 17 Jul 2021 04:39:37 GMT

  # content-type: text/html

  # content-length: 37

  # last-modified: Thu, 15 Jul 2021 12:24:55 GMT

  # etag: "60f02917-25"

  # accept-ranges: bytes

  # strict-transport-security: max-age=15724800; includeSubDomains



  # hostname-web-front-1-WeiyiGeek-Nginx



$ curl 'https://10.10.107.221/tomcat' -H 'host: test.weiyigeek.top' -i -k

  # HTTP/2 200

  # date: Sat, 17 Jul 2021 04:39:50 GMT

  # content-type: text/html

  # content-length: 40

  # accept-ranges: bytes

  # etag: W/"40-1626351892000"

  # last-modified: Thu, 15 Jul 2021 12:24:52 GMT

  # strict-transport-security: max-age=15724800; includeSubDomains



  # hostname-web-backend-0-WeiyiGeek-Tomcat
WeiyiGeek.Ingress 应用
WeiyiGeek.Ingress 应用

0x02 入坑出坑

问题1.K8s Nginx Ingress Controller 转发 X-Forwarded-Proto 请求头的问题

问题说明: 在 Kubernetes 集群上部署了 Nginx Ingress Controller 最前端用的是阿里云七层负载均衡,部署后发现不能正确转发 X-Forwarded-Proto 请求头,造成 http 重定向到 https 无法正常工作,请问如何解决?

代码语言:txt
复制
# 问题补充:

# 用下面的命令进入 nginx-ingress 容器

kubectl exec -it daemonset/nginx-ingress -n nginx-ingress cat /etc/nginx/conf.d/production-cnblogs-ingress.conf



# 发现问题是下面的配置引起的

proxy\_set\_header X-Forwarded-Proto $scheme;

问题解决: 终于在 Nginx Ingress Controller 的官方帮助文档 Advanced Configuration with Annotations 中找到一个注解(annotation)解决了这个问题,它就是 nginx.org/redirect-to-https: "true"

代码语言:txt
复制
Annotation: nginx.org/redirect-to-https

ConfigMap Key redirect-to-https

Description Sets the 301 redirect rule based on the value of the http\_x\_forwarded\_proto header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of the Ingress controlle

# 根据服务器块上http\_x\_forwarded\_proto报头的值设置301重定向规则,以强制传入的流量使用HTTPS。在入口控制器前的负载均衡器中终止SSL时非常有用;

操作步骤:

1)在 cnblogs-ingress.yaml 中 annotations 下面添加 nginx.org/redirect-to-https: "true"

代码语言:txt
复制
apiVersion: extensions/v1beta1 

kind: Ingress

metadata:

  name: cnblogs-ingress

  annotations:

    kubernetes.io/ingress.class: nginx

    nginx.org/redirect-to-https: "true"

spec:

  rules:

  - host: q.cnblogs.com

    http:

      paths:

        - backend:

            serviceName: q-web

            servicePort: 80

2) 更新 ingress 配置

代码语言:txt
复制
kubectl apply -f cnblogs-ingress.yaml

3) 更新 nginx-ingress

代码语言:txt
复制
kubectl rollout restart daemonset/nginx-ingress -n nginx-ingress && \

kubectl rollout status daemonset/nginx-ingress -n nginx-ingress

4)查看 inginx 容器中的配置

代码语言:txt
复制
kubectl exec -it daemonset/nginx-ingress -n nginx-ingress cat /etc/nginx/conf.d/production-cnblogs-ingress.conf



# 发现 proxy\_set\_header X-Forwarded-Proto $scheme; 变成了 proxy\_set\_header X-Forwarded-Proto https; ,并且增加了下面的 http 重定向 https 的配置。



if ($http\_x\_forwarded\_proto = 'http') {

    return 301 https://$host$request\_uri;

}

nginx-ingress 自己完成了基于X-Forwarded-Proto 的 http 重定向到 https 的操作,应用都不需要自己处理了。

Tips :建议采用 kubernetes/ingress-nginx 而非 nginxinc/kubernetes-ingress : https://github.com/kubernetes/ingress-nginx

问题2:extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

解决办法:根据当前 Kubernetes 版本中资源组与版本进行选择即可;

代码语言:txt
复制
~/K8s/Day7/demo1$ kubectl api-resources | grep "ingresses"

ingresses                         ing          extensions                     true         Ingress

ingresses                         ing          networking.k8s.io              true         Ingress



# Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

apiVersion: networking.k8s.io/v1 # 注意点否则将报上面的预警
问题3.Error from server (InternalError): error when applying patch:

错误信息:

代码语言:txt
复制
$ kubectl apply -f demo-ingress.yaml

# 错误1

Resource: "networking.k8s.io/v1, Resource=ingresses", GroupVersionKind: "networking.k8s.io/v1, Kind=Ingress"

Name: "nginx", Namespace: "default"

for: "demo-ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s": x509: certificate is valid for k8s-master002, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, not 

ingress-nginx-controller-admission.ingress-nginx.svc



# 错误2

kubectl create -f ingress-nginx-http-v1.yaml

  # Error from server (InternalError): error when creating "ingress-nginx-http-v1.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": an error on the server ("") has prevented the request from succeeding

解决办法: Webhook 删除后重新构建 ingress-nginx-http-v1.yaml 资源清单即可

代码语言:txt
复制
weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission

  # validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted



weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-http-v1.yaml  

  # ingress.networking.k8s.io/nginx-ingress-http created  # 未其他报错正常创建
问题4.查看ingress规则时提示kubernetes之ingress error: endpoints "default-http-backend" not found警告
  • 问题信息:
代码语言:txt
复制
$ kubectl describe ing

Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
  • 问题原因: 注意:根据您正在使用的Ingress控制器,您可能需要创建一个default-http-backend服务。没有规则的入口将所有流量发送到一个默认后端。默认后端通常是Ingress控制器的一个配置选项,在您的Ingress资源中没有指定。如果Ingress对象中的主机或路径都不匹配HTTP请求,则流量将被路由到默认后端。
  • 解决办法:
代码语言:txt
复制
# Ingress 控制器

spec:

  defaultBackend:

    service:

      name: nginx-ingress-v1-svc

      port:

        number: 80 

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。

如有侵权,请联系 cloudcommunity@tencent.com 删除。

评论
登录后参与评论
0 条评论
热度
最新
推荐阅读
目录
  • 示例8.Ingress Header 请求头匹配
  • 示例9.Ingress 灰度金丝雀发布
  • 示例10.Ingress 相关配置补充
  • 企业实战
    • 1.利用Ingress访问后端的Nginx Web和Tomcat后端综合实践。
      • 问题1.K8s Nginx Ingress Controller 转发 X-Forwarded-Proto 请求头的问题
      • 问题2:extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
      • 问题3.Error from server (InternalError): error when applying patch:
      • 问题4.查看ingress规则时提示kubernetes之ingress error: endpoints "default-http-backend" not found警告
  • 0x02 入坑出坑
相关产品与服务
容器服务
腾讯云容器服务(Tencent Kubernetes Engine, TKE)基于原生 kubernetes 提供以容器为核心的、高度可扩展的高性能容器管理服务,覆盖 Serverless、边缘计算、分布式云等多种业务部署场景,业内首创单个集群兼容多种计算节点的容器资源管理模式。同时产品作为云原生 Finops 领先布道者,主导开源项目Crane,全面助力客户实现资源优化、成本控制。
领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档