如下基础示例请看上一篇文章
示例1.Ingress 常规使用方案
示例2.Ingress HTTPS 代理访问
示例3.Ingress Rewrite 重写重定向访问
示例4.Ingress VirtualHost 虚拟主机访问
示例5.Ingress BasicAuth(基础认证)功能
示例6.Ingress Redirect 域名重定向功能
示例7.Ingress 黑白名单访问限制
更多学习笔记文章请关注 WeiyiGeek
公众账号,学习交流【邮箱联系: Master#weiyigeek.top】
原文地址: https://mp.weixin.qq.com/s/ZxyNCP32BQjQo5ygztJttQ
描述: Ingress-nginx 的 匹配请求头,可以采用nginx.ingress.kubernetes.io/server-snippet
注解。
# - 资源清单
tee ingress-header-request.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-heade
namespace: demo
annotations:
nginx.ingress.kubernetes.io/server-snippet: |-
set $agentflag 0;
if ($http\_user\_agent ~\* "(Firefox)" ){
set $agentflag 1;
}
if ($agentflag = 1) {
return 301 https://www.weiyigeek.top;
}
spec:
rules:
- host: header.weiyigeek.top
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: backend-svc
port:
number: 8080
status:
loadBalancer: {}
EOF
# - 创建查看 Ingress 资源对象
kubectl create -f ingress-header-request.yaml
kubectl get ing -n demo ingress-header && kubectl describe ing -n demo ingress-heade
# Rules:
# Host Path Backends
# ---- ---- --------
# header.weiyigeek.top / backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)
# - 验证 Ingress 资源对象
# (1) 指定 User-Agent 请求头 包含 Firefox
$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0' -k -i
# HTTP/1.1 301 Moved Permanently
# Date: Fri, 16 Jul 2021 14:00:04 GMT
# Content-Type: text/html
# Content-Length: 162
# Connection: keep-alive
# Location: https://www.weiyigeek.top
# <html>
# <head><title>301 Moved Permanently</title></head>
# <body>
# <center><h1>301 Moved Permanently</h1></center>
# <hr><center>nginx</center>
# </body>
# </html>
# (2) 指定 User-Agent 请求头 不包含 Firefox
$ curl http://10.10.107.221/ -H 'host: header.weiyigeek.top' -H 'User-Agent: Mozilla/5.0' -k -i
# HTTP/1.1 200
# Date: Fri, 16 Jul 2021 14:03:29 GMT
# Content-Type: text/html
# Content-Length: 40
# Connection: keep-alive
# Accept-Ranges: bytes
# ETag: W/"40-1626351892000"
# Last-Modified: Thu, 15 Jul 2021 12:24:52 GMT
# hostname-web-backend-0-WeiyiGeek-Tomcat
描述: 在某些情况下,您可能希望通过向与生产服务不同的服务发送少量请求来“金丝雀”一组新的更改。金丝雀注解使 Ingress 规范能够根据应用的规则充当路由请求的替代服务,在nginx.ingress.kubernetes.io/canary: "true"
设置后可以启用以下用于配置金丝雀的注释:。
Nginx Annotations 支持以下 4 种 Canary 规则:
\* nginx.ingress.kubernetes.io/canary-weight:基于服务权重的流量切分,适用于蓝绿部署,权重范围 0 - 100 按百分比将请求路由到 Canary Ingress 中指定的服务。权重为 0 意味着该金丝雀规则不会向 Canary 入口的服务发送任何请求。权重为 100 意味着所有请求都将被发送到 Canary 入口。
\* nginx.ingress.kubernetes.io/canary-by-header:基于 Request Header 的流量切分,适用于灰度发布以及 A/B 测试。当 Request Header 设置为 always时,请求将会被一直发送到 Canary 版本;当 Request Header 设置为 never时,请求不会被发送到 Canary 入口;对于任何其他 Header 值,将忽略 Header,并通过优先级将请求与其他金丝雀规则进行优先级的比较。
\* nginx.ingress.kubernetes.io/canary-by-header-value:要匹配的 Request Header 的值,用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务。当 Request Header 设置为此值时,它将被路由到 Canary 入口。该规则允许用户自定义 Request Header 的值,必须与上一个 annotation (即:canary-by-header)一起使用。
\* nginx.ingress.kubernetes.io/canary-by-header-pattern:这与canary-by-header-valuePCRE 正则表达式匹配的工作方式相同。请注意,canary-by-header-value设置此注释时将被忽略。当给定的 Regex 在请求处理过程中导致错误时,该请求将被视为不匹配。
\* nginx.ingress.kubernetes.io/canary-by-cookie:基于 Cookie 的流量切分,适用于灰度发布与 A/B 测试。用于通知 Ingress 将请求路由到 Canary Ingress 中指定的服务的cookie。当 cookie 值设置为 always时,它将被路由到 Canary 入口;当 cookie 值设置为 never时,请求不会被发送到 Canary 入口;对于任何其他值,将忽略 cookie 并将请求与其他金丝雀规则进行优先级的比较。 定义两个版本的代码。
Tips : 金丝雀规则是按优先顺序计算的。优先级如下:canary by header->canary by cookie->canary weight
Tips : 注意当您将入口标记为 Canary 时,除nginx.ingress.kubernetes.io/load-balanceand
之外的所有其他非 Canary 注释都将被忽略(从相应的主入口继承)nginx.ingress.kubernetes.io/upstream-hash-by
。`
**资源配置清单:**
# - 默认访问前端
tee ingress-canary-v1.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-canary-v1
namespace: demo
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: canary
nginx.ingress.kubernetes.io/canary-by-header-value: v1
nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
rules:
- host: canary.weiyigeek.top
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: front-svc
port:
number: 80
EOF
# - 利用金丝雀访问后端
tee ingress-canary-v2.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-canary-v2
namespace: demo
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-by-header: canary
nginx.ingress.kubernetes.io/canary-by-header-value: v2
nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
rules:
- host: canary.weiyigeek.top
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: backend-svc
port:
number: 8080
EOF
~/k8s/ingress/demo1# kubectl create -f ingress-canary-v1.yaml
# ingress.networking.k8s.io/ingress-canary-v1 created
~/k8s/ingress/demo1# kubectl create -f ingress-canary-v2.yaml
# ingress.networking.k8s.io/ingress-canary-v2 created
~/k8s/ingress/demo1# kubectl get ing -n demo ingress-canary-v1 ingress-canary-v2
# NAME CLASS HOSTS ADDRESS PORTS AGE
# ingress-canary-v1 <none> canary.weiyigeek.top 80 3m12s
# ingress-canary-v2 <none> canary.weiyigeek.top 80 3m4s
# 验证金丝雀 Ingress 对象应用
for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -i;done
for i in {1..3};do curl http://10.10.107.221/ -H 'host: canary.weiyigeek.top' -H 'weiyigeek: v2' -i;done
描述: 可以修改 deploy.yaml 中指定 nginx-ingress-controller 运行的参数,在此处添加- --default-backend-service=名称空间/SVC名称
参数
$ grep -A 3 -n "/nginx-ingress" deploy.yaml
332: - /nginx-ingress-controlle
333- - --publish-service=$(POD\_NAMESPACE)/ingress-nginx-controlle
334- - --election-id=ingress-controller-leade
335- - --ingress-class=nginx
- --default-backend-service=demo/error-svc
描述: 下述注释定义了对连接和传输速率的限制,这些可以用来减轻DDoS攻击。
\* nginx.ingress.kubernetes.io/limit-connections:单个IP地址允许的并发连接数。超出此限制时,将返回503错误。
\* nginx.ingress.kubernetes.io/limit-rps:每秒从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。
\* nginx.ingress.kubernetes.io/limit-rpm:每分钟从给定IP接受的请求数。突发限制设置为此限制乘以突发乘数,默认乘数为5。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。
\* nginx.ingress.kubernetes.io/limit-burst-multiplier:突发大小限制速率的倍数。默认的脉冲串乘数为5,此注释将覆盖默认的乘数。当客户端超过此限制时,将 返回limit-req-status-code默认值: 503。
\* nginx.ingress.kubernetes.io/limit-rate-after:最初的千字节数,在此之后,对给定连接的响应的进一步传输将受到速率的限制。必须在启用代理缓冲的情况下使用此功能。
\* nginx.ingress.kubernetes.io/limit-rate:每秒允许发送到给定连接的千字节数。零值禁用速率限制。必须在启用代理缓冲的情况下使用此功能。
\* nginx.ingress.kubernetes.io/limit-whitelist:客户端IP源范围要从速率限制中排除。该值是逗号分隔的CIDR列表。
**示例说明:**
# - 资源清单配置:
tee ingress-limit.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-limit
namespace: demo
annotations:
nginx.ingress.kubernetes.io/limit-rate: "100K" # 限制客户端每秒传输的字节数
nginx.ingress.kubernetes.io/limit-whitelist: "10.10.107.220" # 白名单中的IP不限速
nginx.ingress.kubernetes.io/limit-rps: 1 # 单个IP每秒的连接数
nginx.ingress.kubernetes.io/limit-rpm: 30 # 单个IP每分钟的连接数
spec:
rules:
- host: limit.weiyigeek.top
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: backend-svc
port:
number: 8080
status:
loadBalancer: {}
EOF
描述: 前面我们说到我们除了可以采用annotations方式配置ingress规则外也可以通过ConfigMap来存储相应的Ingress规则;
Tips : 通过 Ingress ConfigMaps 查看详细的配置参数。
Ingress 规则存储配置清单:
# --configmap=
tee nginx-config.yaml <<'EOF'
kind: ConfigMap
apiVersion: v1
metadata:
name: ingress-nginx-controller # 注意此ConfigMap名称与/nginx-ingress-controller 中 `--configmap=$(POD\_NAMESPACE)/ingress-nginx-controller` 对应
namespace: ingress-nginx
labels:
app: ingress-nginx
data:
proxy-connect-timeout: "15" #这里有个坑,到家一定要注意是'-'不是'\_'
EOF
# --tcp-services-configmap=
---
kind: ConfigMap
apiVersion: v1
metadata:
name: tcp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
# --udp-services-configmap
---
kind: ConfigMap
apiVersion: v1
metadata:
name: udp-services
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
nginx-ingress-controller 使用 Configmap 作为配置存在控制器:
# 可以在 nginx-ingress-controller 部署资源清单中看见`--configmap=`参数
vim +328 deploy.yaml
containers:
- name: controlle
image: pollyduan/ingress-nginx-controller:v0.47.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
args:
- /nginx-ingress-controlle
- --publish-service=$(POD\_NAMESPACE)/ingress-nginx-controlle
- --election-id=ingress-controller-leade
- --ingress-class=nginx
- --configmap=$(POD\_NAMESPACE)/ingress-nginx-controller # 使用 confimap 配置文件
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
查看ingress-nginx名称空间中的ConfigMap配置
kubectl apply -f nginx-config.yaml
# configmap/ingress-nginx-controller configured
kubectl get cm -n ingress-nginx
# NAME DATA AGE
# ingress-controller-leader-nginx 0 2d8h
# ingress-nginx-controller 0 2d8h
# kube-root-ca.crt 1 2d8h
kubectl describe cm -n ingress-nginx ingress-nginx-controlle
# Name: ingress-nginx-controlle
# Namespace: ingress-nginx
# Labels: app.kubernetes.io/component=controlle
# app.kubernetes.io/instance=ingress-nginx
# app.kubernetes.io/managed-by=Helm
# app.kubernetes.io/name=ingress-nginx
# app.kubernetes.io/version=0.47.0
# helm.sh/chart=ingress-nginx-3.33.0
# Annotations: <none>
# Data
# ====
# proxy-connect-timeout:
# ----
# 15
# Events:
# Type Reason Age From Message
# ---- ------ ---- ---- -------
# Normal UPDATE 25s nginx-ingress-controller ConfigMap ingress-nginx/ingress-nginx-controlle
验证是否在ingress-nginx-controller中配置生效:
kubectl get pod -n ingress-nginx -l app.kubernetes.io/component=controlle
# NAME READY STATUS RESTARTS AGE
# ingress-nginx-controller-5df4f79b95-6p8rr 1/1 Running 0 2d8h
# 在 ingress-nginx-controller-5df4f79b95-6p8r 的 Pod 中 查看 nginx.conf 文件中 proxy\_connect\_timeout 参数是否为 15s
root@k8s-master-1:~/k8s/ingress/deployment# kubectl -n ingress-nginx exec -it ingress-nginx-controller-5df4f79b95-6p8rr bash
bash-5.1$ grep "proxy\_connect\_timeout" nginx.conf
proxy\_connect\_timeout 15s; #
**实验目标:**
test.weiyigeek.top
以及demo.weiyigeek.top
证书以供Ingress使用。Nginx Web
与Tomcat 后端
应用。/
为NginxWeb而/tomcat
Tomcat 后端需要BasicAuth(基础认证)访问功能**详细流程:**
Nginx Web
与Tomcat 后端
应用以及创建名称空间。tee sts-svc-nginx-tomcat.yaml<<'EOF'
apiVersion: v1
kind: Namespace
metadata:
name: demo
---
# Service 服务创建(服务是软件服务的命名抽象)。
# kubectl api-resources | grep "service"
apiVersion: v1
kind: Service
metadata:
name: front-svc
namespace: demo
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
selector:
app: nginx-web
ports:
- port: 80
targetPort: 80
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: backend-svc
namespace: demo
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
selector:
app: tomcat-web
ports:
- port: 8080
targetPort: 8080
protocol: TCP
sessionAffinity: None
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-front
namespace: demo
spec:
replicas: 2
serviceName: front-svc
selector:
matchLabels:
app: nginx-web
template:
metadata:
labels:
app: nginx-web
spec:
initContainers:
- name: init-html
image: busybox:latest
imagePullPolicy: IfNotPresent
command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/share/nginx/html/index.html"]
env:
- name: MSG
value: "WeiyiGeek-Nginx"
volumeMounts:
- name: web
mountPath: "/usr/share/nginx/html"
securityContext:
privileged: true
containers:
- name: nginx
image: nginx:latest
imagePullPolicy: IfNotPresent
env:
- name: TZ
value: "Asia/Shanghai"
ports:
- name: nginx
containerPort: 80
volumeMounts:
- name: web
mountPath: "/usr/share/nginx/html"
volumes:
- name: web
emptyDir: {}
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web-backend
namespace: demo
spec:
replicas: 2
serviceName: backend-svc
selector:
matchLabels:
app: tomcat-web
template:
metadata:
labels:
app: tomcat-web
spec:
initContainers:
- name: init-html
image: busybox:latest
imagePullPolicy: IfNotPresent
command: ['sh', '-c', "echo hostname-${HOSTNAME}-${MSG} > /usr/local/tomcat/webapps/ROOT/index.html"]
env:
- name: MSG
value: WeiyiGeek-Tomcat
volumeMounts:
- name: web
mountPath: "/usr/local/tomcat/webapps/ROOT/"
securityContext:
privileged: true
containers:
- name: nginx
image: "tomcat:8.5.69-jdk8-openjdk-slim-buster"
imagePullPolicy: IfNotPresent
env:
- name: TZ
value: "Asia/Shanghai"
ports:
- name: nginx
containerPort: 8080
volumeMounts:
- name: web
mountPath: "/usr/local/tomcat/webapps/ROOT/"
volumes:
- name: web
emptyDir: {}
EOF
# 利用资源清单进行创建nginx、Tomcat应用
kubectl create -f sts-svc-nginx-tomcat.yaml
# namespace/demo created
# service/front-svc created
# service/backend-svc created
# statefulset.apps/web-front created
# statefulset.apps/web-backend created
# 查看 Service 控制器对象
$ kubectl get svc -n demo -o wide
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
# backend-svc ClusterIP None <none> 8080/TCP 12m name=tomcat-web
# front-svc ClusterIP None <none> 80/TCP 12m name=nginx-web
# 查看 StatefulSet 控制器对象
$ kubectl get sts -n demo -o wide
# NAME READY AGE CONTAINERS IMAGES
# web-backend 2/2 13m nginx tomcat:8.5.69-jdk8-openjdk-slim-buste
# web-front 2/2 13m nginx nginx:latest
# 查看 nginx 与 tomcat 相关 Pod 信息
$ kubectl get pod -n demo -o wide --show-labels
# NAME READY STATUS RESTARTS AGE IP NODE LABELS
# web-backend-0 1/1 Running 0 13m 192.168.109.90 k8s-node-1 app=tomcat-web,controller-revision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-0
# web-backend-1 1/1 Running 0 13m 192.168.109.93 k8s-node-1 app=tomcat-web,controller-revision-hash=web-backend-776459fc9d,statefulset.kubernetes.io/pod-name=web-backend-1
# web-front-0 1/1 Running 0 13m 192.168.109.91 k8s-node-1 app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-0
# web-front-1 1/1 Running 0 13m 192.168.109.92 k8s-node-1 app=nginx-web,controller-revision-hash=web-front-89f7fcc75,statefulset.kubernetes.io/pod-name=web-front-1
# 查看 SVC 服务发现利用CoreDNS进行域名解析
$ nslookup front-svc.demo.svc.cluster.local. 10.96.0.10
# Server: 10.96.0.10
# Address: 10.96.0.10#53
# Name: front-svc.demo.svc.cluster.local
# Address: 192.168.109.91
# Name: front-svc.demo.svc.cluster.local
# Address: 192.168.109.92
$ nslookup backend-svc.demo.svc.cluster.local. 10.96.0.10
# ...
# Name: backend-svc.demo.svc.cluster.local
# Address: 192.168.109.90
# Name: backend-svc.demo.svc.cluster.local
# 请求查看 Nginx 与 Tomcat 应用回显。
curl front-svc.demo.svc.cluster.local
# hostname-web-front-0-WeiyiGeek-Nginx
curl front-svc.demo.svc.cluster.local
# hostname-web-front-1-WeiyiGeek-Nginx
curl backend-svc.demo.svc.cluster.local:8080
# hostname-web-backend-1-WeiyiGeek-Tomcat
curl backend-svc.demo.svc.cluster.local:8080
# hostname-web-backend-0-WeiyiGeek-Tomcat
# Tips: kubectl api-resources | grep "secret" 可查看apiversions以及namespace是否支持
# 假设你已经将证书上传到k8s集群目录中并解压
$ ls
demo.weiyigeek.top.zip test.weiyigeek.top.zip
$ unzip demo.weiyigeek.top.zip -d demo.weiyigeek.top
$ unzip test.weiyigeek.top.zip -d test.weiyigeek.top
# 创建 tls 类型的 secret
$ cd /tmp/demo.weiyigeek.top/Nginx
kubectl -n demo create secret tls demoweiyigeektop --key 2\_demo.weiyigeek.top.key --cert 1\_demo.weiyigeek.top\_bundle.crt
# secret/demoweiyigeektop created
$ cd /tmp/test.weiyigeek.top/Nginx
kubectl -n demo create secret tls testweiyigeektop --key 2\_test.weiyigeek.top.key --cert 1\_test.weiyigeek.top\_bundle.crt
# secret/testweiyigeektop created
tee ingress-demo.yaml <<'EOF'
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-demo
namespace: demo
annotations:
# 指定 Ingress Controller 的类型
kubernetes.io/ingress.class: "nginx"
# 指定我们的 rules 的 path 可以使用正则表达式
nginx.ingress.kubernetes.io/use-regex: "true"
# 连接超时时间,默认为 5s
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
# 后端服务器回转数据超时时间,默认为 60s
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
# 后端服务器响应超时时间,默认为 60s
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
# 客户端上传文件,最大大小,默认为 20m
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
# URL 重写
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
# 默认后端服务指定
defaultBackend:
service:
name: front-svc
port:
number: 80
tls:
- hosts:
- test.weiyigeek.top
secretName: testweiyigeektop
- hosts:
- demo.weiyigeek.top
secretName: demoweiyigeektop
rules:
- host: test.weiyigeek.top
http:
paths:
- pathType: Prefix
path: /tomcat
backend:
service:
name: backend-svc
port:
number: 8080
- pathType: Prefix
path: /nginx
backend:
service:
name: front-svc
port:
number: 80
- host: demo.weiyigeek.top
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: front-svc
port:
number: 80
EOF
$ kubectl create -f ingress-demo.yaml
# ingress.networking.k8s.io/ingress-demo created
# 查看 ingress-nginx 的 SVC
kubectl get svc -n ingress-nginx
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# ingress-nginx-controller LoadBalancer 10.104.171.212 <pending> 80:31122/TCP,443:32155/TCP 24h
# ingress-nginx-controller-admission ClusterIP 10.99.168.221 <none> 443/TCP 24h
# 查看 ingress-demo 对象相关信息
$ kubectl get ing -n demo ingress-demo
# NAME CLASS HOSTS ADDRESS PORTS AGE
# ingress-demo <none> test.weiyigeek.top,demo.weiyigeek.top 80, 443 24h
$ kubectl describe ing -n demo ingress-demo
# Name: ingress-demo
# Namespace: demo
# Address:
# Default backend: front-svc:80 (192.168.109.91:80,192.168.109.92:80)
# TLS:
# testweiyigeektop terminates test.weiyigeek.top
# demoweiyigeektop terminates demo.weiyigeek.top
# Rules:
# Host Path Backends
# ---- ---- --------
# test.weiyigeek.top /tomcat backend-svc:8080 (192.168.109.90:8080,192.168.109.93:8080)
# demo.weiyigeek.top / front-svc:80 (192.168.109.91:80,192.168.109.92:80)
# Annotations: kubernetes.io/ingress.class: nginx
# nginx.ingress.kubernetes.io/proxy-body-size: 10m
# nginx.ingress.kubernetes.io/proxy-connect-timeout: 600
# nginx.ingress.kubernetes.io/proxy-read-timeout: 600
# nginx.ingress.kubernetes.io/proxy-send-timeout: 600
# nginx.ingress.kubernetes.io/rewrite-target: /
# nginx.ingress.kubernetes.io/use-regex: true
# Events: <none>
https://demo.weiyigeek.top
与 https://test.weiyigeek.top
站点。$ curl 'https://10.10.107.221' -H 'host: demo.weiyigeek.top' -i -k
# HTTP/2 200
# date: Sat, 17 Jul 2021 04:39:37 GMT
# content-type: text/html
# content-length: 37
# last-modified: Thu, 15 Jul 2021 12:24:55 GMT
# etag: "60f02917-25"
# accept-ranges: bytes
# strict-transport-security: max-age=15724800; includeSubDomains
# hostname-web-front-1-WeiyiGeek-Nginx
$ curl 'https://10.10.107.221/tomcat' -H 'host: test.weiyigeek.top' -i -k
# HTTP/2 200
# date: Sat, 17 Jul 2021 04:39:50 GMT
# content-type: text/html
# content-length: 40
# accept-ranges: bytes
# etag: W/"40-1626351892000"
# last-modified: Thu, 15 Jul 2021 12:24:52 GMT
# strict-transport-security: max-age=15724800; includeSubDomains
# hostname-web-backend-0-WeiyiGeek-Tomcat
问题说明: 在 Kubernetes 集群上部署了 Nginx Ingress Controller 最前端用的是阿里云七层负载均衡,部署后发现不能正确转发 X-Forwarded-Proto 请求头,造成 http 重定向到 https 无法正常工作,请问如何解决?
# 问题补充:
# 用下面的命令进入 nginx-ingress 容器
kubectl exec -it daemonset/nginx-ingress -n nginx-ingress cat /etc/nginx/conf.d/production-cnblogs-ingress.conf
# 发现问题是下面的配置引起的
proxy\_set\_header X-Forwarded-Proto $scheme;
问题解决: 终于在 Nginx Ingress Controller 的官方帮助文档 Advanced Configuration with Annotations 中找到一个注解(annotation)解决了这个问题,它就是 nginx.org/redirect-to-https: "true"
Annotation: nginx.org/redirect-to-https
ConfigMap Key redirect-to-https
Description Sets the 301 redirect rule based on the value of the http\_x\_forwarded\_proto header on the server block to force incoming traffic to be over HTTPS. Useful when terminating SSL in a load balancer in front of the Ingress controlle
# 根据服务器块上http\_x\_forwarded\_proto报头的值设置301重定向规则,以强制传入的流量使用HTTPS。在入口控制器前的负载均衡器中终止SSL时非常有用;
操作步骤:
1)在 cnblogs-ingress.yaml 中 annotations 下面添加 nginx.org/redirect-to-https: "true"
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.org/redirect-to-https: "true"
spec:
rules:
- host: q.cnblogs.com
http:
paths:
- backend:
serviceName: q-web
servicePort: 80
2) 更新 ingress 配置
kubectl apply -f cnblogs-ingress.yaml
3) 更新 nginx-ingress
kubectl rollout restart daemonset/nginx-ingress -n nginx-ingress && \
kubectl rollout status daemonset/nginx-ingress -n nginx-ingress
4)查看 inginx 容器中的配置
kubectl exec -it daemonset/nginx-ingress -n nginx-ingress cat /etc/nginx/conf.d/production-cnblogs-ingress.conf
# 发现 proxy\_set\_header X-Forwarded-Proto $scheme; 变成了 proxy\_set\_header X-Forwarded-Proto https; ,并且增加了下面的 http 重定向 https 的配置。
if ($http\_x\_forwarded\_proto = 'http') {
return 301 https://$host$request\_uri;
}
nginx-ingress 自己完成了基于X-Forwarded-Proto
的 http 重定向到 https 的操作,应用都不需要自己处理了。
Tips :建议采用 kubernetes/ingress-nginx 而非 nginxinc/kubernetes-ingress : https://github.com/kubernetes/ingress-nginx
解决办法:根据当前 Kubernetes 版本中资源组与版本进行选择即可;
~/K8s/Day7/demo1$ kubectl api-resources | grep "ingresses"
ingresses ing extensions true Ingress
ingresses ing networking.k8s.io true Ingress
# Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
apiVersion: networking.k8s.io/v1 # 注意点否则将报上面的预警
错误信息:
$ kubectl apply -f demo-ingress.yaml
# 错误1
Resource: "networking.k8s.io/v1, Resource=ingresses", GroupVersionKind: "networking.k8s.io/v1, Kind=Ingress"
Name: "nginx", Namespace: "default"
for: "demo-ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/extensions/v1beta1/ingresses?timeout=30s": x509: certificate is valid for k8s-master002, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, not
ingress-nginx-controller-admission.ingress-nginx.svc
# 错误2
kubectl create -f ingress-nginx-http-v1.yaml
# Error from server (InternalError): error when creating "ingress-nginx-http-v1.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": an error on the server ("") has prevented the request from succeeding
解决办法: Webhook 删除后重新构建 ingress-nginx-http-v1.yaml 资源清单即可
weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission
# validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted
weiyigeek@ubuntu:~/K8s/Day7/demo2$ kubectl create -f ingress-nginx-http-v1.yaml
# ingress.networking.k8s.io/nginx-ingress-http created # 未其他报错正常创建
kubernetes之ingress error: endpoints "default-http-backend" not found
警告$ kubectl describe ing
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
# Ingress 控制器
spec:
defaultBackend:
service:
name: nginx-ingress-v1-svc
port:
number: 80
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。