K8s反向代理负载均衡组件ingress
K8s反向代理负载均衡组件ingress
参考文档
https://github.com/kubernetes/ingress/tree/master/examples
https://mritd.me/2017/03/04/how-to-use-nginx-ingress/
http://www.dockerinfo.net/1132.html
k8s集群安装部署
http://jerrymin.blog.51cto.com/3002256/1898243
k8s集群RC、SVC、POD部署
http://jerrymin.blog.51cto.com/3002256/1900260
k8s集群组件kubernetes-dashboard和kube-dns部署
http://jerrymin.blog.51cto.com/3002256/1900508
k8s集群监控组件heapster部署
http://jerrymin.blog.51cto.com/3002256/1904460
k8s集群反向代理负载均衡组件部署
http://jerrymin.blog.51cto.com/3002256/1904463
k8s集群挂载volume之nfs
http://jerrymin.blog.51cto.com/3002256/1906778
k8s集群挂载volume之glusterfs
http://jerrymin.blog.51cto.com/3002256/1907274
架构
Ingress是对外（公网）服务到集群内的Service之间规则的集合：允许进入集群的请求被转发至集群内的Service，过程类似下图：
部署步骤
1，下载组件ingress源代码
[root@k8s-master ~]# wgethttps://github.com/kubernetes/ingress/archive/nginx-0.9.0-beta.2.tar.gz
[root@k8s-master~]# cd ingress-nginx-0.9.0-beta.2/
[root@k8s-masteringress-nginx-0.9.0-beta.2]# ls
CONTRIBUTING.md controllers core docs examples Godeps hack images LICENSE Makefile OWNERS README.md vendor
2，找到安装方法，一般在部署目录
[root@k8s-masternginx]# pwd
/root/ingress-nginx-0.9.0-beta.2/examples/deployment/nginx
[root@k8s-masternginx]# ls
default-backend.yaml kubeadm nginx-ingress-controller.yaml README.md
[root@k8s-masternginx]# cat README.md
#Deploying the Nginx Ingress controller
Thisexample aims to demonstrate the deployment of an nginx ingress controller.
##Default Backend
The defaultbackend is a Service capable of handling all url paths and hosts the
nginxcontroller doesn't understand. This most basic implementation just returns
a 404page:
```console
$kubectl apply -f default-backend.yaml
deployment"default-http-backend" created
service"default-http-backend" created
$kubectl -n kube-system get po
NAME READY STATUS RESTARTS AGE
default-http-backend-2657704409-qgwdd 1/1 Running 0 28s
```
##Controller
You candeploy the controller as follows:
```console
$kubectl apply -f nginx-ingress-controller.yaml
deployment"nginx-ingress-controller" created
$kubectl -n kube-system get po
NAME READY STATUS RESTARTS AGE
default-http-backend-2657704409-qgwdd 1/1 Running 0 2m
nginx-ingress-controller-873061567-4n3k2 1/1 Running 0 42s
```
Notethe default settings of this controller:
*serves a `/healthz` url on port 10254, as both a liveness and readiness probe
*takes a `--default-backend-service` argument pointing to the Service createdabove
3，在节点上先下载需要的镜像
[root@k8s-node1~]# docker images|grep ingress
gcr.io/google_containers/nginx-ingress-controller 0.9.0-beta.2 c465518591e5 12 days ago 121.1 MB
4，按照ReadME介绍方法安装部署
[root@k8s-masternginx]# kubectl apply -f default-backend.yaml
deployment"default-http-backend" created
service"default-http-backend" created
[root@k8s-masternginx]# kubectl apply -f nginx-ingress-controller.yaml
deployment"nginx-ingress-controller" created
[root@k8s-masternginx]# kubectl -n kube-system get po
NAME READY STATUS RESTARTS AGE
default-http-backend-2657704409-jltgk 1/1 Running 0 12s
heapster-791010959-78jz8 1/1 Running 0 14h
kube-dns-3019842428-fkgh5 3/3 Running 3 4d
kube-dns-autoscaler-2715466192-q0t0c 1/1 Running 1 4d
kubernetes-dashboard-47555765-2w64l 1/1 Running 1 4d
monitoring-grafana-3730655072-gq4h9 1/1 Running 0 14h
monitoring-influxdb-957705310-424kg 1/1 Running 0 14h
nginx-ingress-controller-3752011415-xj5rr 0/1 Running 0 6s
5，创建测试Ingress记录
先创建规则，命名空间kube-system前期安装了dashboard和监控，虽然可以通过nodeport发布，现在试试ingress
[root@k8s-masternginx]# cat k8s.yaml
apiVersion:extensions/v1beta1
kind:Ingress
metadata:
name: dashboard-monitor-ingress
namespace: kube-system
spec:
rules:
- host: dashboard.test.com
http:
paths:
- backend:
serviceName: kubernetes-dashboard
servicePort: 80
- host: monitor.test.com
http:
paths:
- backend:
[root@k8s-masternginx]# kubectl create -f k8s.yaml
ingress"dashboard-monitor-ingress" created
后进入容器查看配置，下面贴出的都是Ingress自动添加的
[root@k8s-masternginx]# kubectl exec -ti nginx-ingress-controller-3752011415-xj5rr -nkube-system -- bash
root@nginx-ingress-controller-3752011415-xj5rr:/#cat /etc/nginx/nginx.conf
# In case of errors try the next upstreamserver before returning an error
proxy_next_upstream error timeoutinvalid_header http_502 http_503 http_504;
upstreamkube-system-kubernetes-dashboard-80 {
least_conn;
server 10.1.15.7:9090 max_fails=0fail_timeout=0;
}
upstream kube-system-monitoring-grafana-80{
least_conn;
server 10.1.39.6:3000 max_fails=0fail_timeout=0;
}
server {
server_name dashboard.test.com;
listen [::]:80;
location / {
set $proxy_upstream_name "kube-system-kubernetes-dashboard-80";
port_in_redirect off;
client_max_body_size "1m";
proxy_set_header Host $host;
# Pass Real IP
proxy_set_header X-Real-IP $remote_addr;
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_headerX-Forwarded-Host $host;
proxy_set_headerX-Forwarded-Port $pass_port;
proxy_set_headerX-Forwarded-Proto $pass_access_scheme;
# mitigate HTTPoxy Vulnerability
#https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_redirect off;
proxy_buffering off;
proxy_buffer_size "4k";
proxy_http_version 1.1;
proxy_passhttp://kube-system-kubernetes-dashboard-80;
}
}
server {
server_name monitor.test.com;
listen [::]:80;
location / {
set $proxy_upstream_name"kube-system-monitoring-grafana-80";
port_in_redirect off;
client_max_body_size "1m";
proxy_set_header Host $host;
# Pass Real IP
proxy_set_header X-Real-IP $remote_addr;
# Allow websocket connections
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_headerX-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_headerX-Forwarded-Host $host;
proxy_set_headerX-Forwarded-Port $pass_port;
proxy_set_headerX-Forwarded-Proto $pass_access_scheme;
# mitigate HTTPoxy Vulnerability
#https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
proxy_set_header Proxy "";
# Custom headers
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_redirect off;
proxy_buffering off;
proxy_buffer_size "4k";
proxy_http_version 1.1;
proxy_passhttp://kube-system-monitoring-grafana-80;
}
}6，测试验证
此时绑定host到ingress所在Node节点的ip就可以访问上面两个域名了，默认节点80端口监控所有的状态
http://monitor.test.com/dashboard/db/cluster
http://dashboard.test.com/#/workload?namespace=default
