专栏首页charlieroroIstio的流量管理(实操二)(istio 系列四)

Istio的流量管理(实操二)(istio 系列四)

Istio的流量管理(实操二)(istio 系列四)

涵盖官方文档Traffic Management章节中的inrgess部分。

目录

Ingress网关

在kubernetes环境中,kubernetes ingress资源用于指定暴露到集群外的服务。在istio服务网格中,使用了一种不同的配置模型,称为istio网关。一个网关允许将istio的特性,如镜像和路由规则应用到进入集群的流量上。

本节描述了如何使用istio网关将一个服务暴露到服务网格外。

环境准备

使用如下命令创建httpbin服务

$ kubectl apply -f samples/httpbin/httpbin.yaml

确定ingress的IP和端口

由于本环境中没有配置对外的负载均衡,因此此处的EXTERNAL-IP为空,使用node port进行访问

# kubectl get svc istio-ingressgateway -n istio-system
NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP  PORT(S)    AGE
istio-ingressgateway   LoadBalancer   10.84.93.45   <pending>     ...        11d

获取ingressgateway service的http2https对应的端口

$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')
$ export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].nodePort}')

下面是istio-system命名空间的istio-ingressgateway service中的一部分端口信息,可以看到http2https的nodeport分别为3119431785,对应上面的INGRESS_PORTSECURE_INGRESS_PORT

{
    "name": "http2",
    "nodePort": 31194,
    "port": 80,
    "protocol": "TCP",
    "targetPort": 80
},
{
    "name": "https",
    "nodePort": 31785,
    "port": 443,
    "protocol": "TCP",
    "targetPort": 443
},

获取istio-system命名空间中ingressgateway pod 的hostIP

$ export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')

使用istio网关配置ingress

一个ingress网关描述了在网格边缘用于接收入站HTTP/TCP连接的负载均衡,配置了暴露的端口,协议等。kubernetes ingress资源不包括任何流量路由配置。ingress 流量的路由使用istio路由规则,与内部服务请求相同:

  1. 创建istio Gateway,将来自httpbin.example.com的流量导入网格的80端口(即默认的ingressgatewaypod) kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: httpbin-gateway spec: selector: istio: ingressgateway # use Istio default gateway implementation servers: - port: number: 80 name: http protocol: HTTP hosts: - "httpbin.example.com" EOF
  2. 通过Gateway配置进入的流量路由,将来自httpbin.example.com,且目的地为/status/delay的请求分发到httpbin服务的8000端口,其他请求会返回404响应。 kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: httpbin spec: hosts: - "httpbin.example.com" gateways: - httpbin-gateway http: - match: - uri: prefix: /status - uri: prefix: /delay route: - destination: port: number: 8000 host: httpbin EOF 可以看到流量被导入了httpbin service暴露的8000端口上 $ oc get svc |grep httpbin httpbin ClusterIP 10.84.222.69 <none> 8000/TCP 19h 来自网格内部其他服务的请求则不受此规则的约束,会使用默认的轮询路由进行请求分发。为了限制内部的调用规则,可以将特定的值mesh添加到gateways列表中。由于内部服务的主机名可能与外部不同,因此需要将主机名添加到hosts列表中。
  3. 使用curl命令访问httpbin服务,此时通过-H选项修改了HTTP请求首部的Host字段,使用http2的nodeport方式访问: $ curl -s -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/status/200 HTTP/1.1 200 OK server: istio-envoy date: Thu, 21 May 2020 03:22:50 GMT content-type: text/html; charset=utf-8 access-control-allow-origin: * access-control-allow-credentials: true content-length: 0 x-envoy-upstream-service-time: 21
  4. 访问其他URL路径则返回404错误 $ curl -s -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/headers HTTP/1.1 404 Not Found date: Thu, 21 May 2020 03:25:20 GMT server: istio-envoy transfer-encoding: chunked

使用浏览器访问ingress服务

由于无法像使用curl一样修改请求的Host首部字段,因此无法使用浏览器访问httpbin服务。为了解决这个问题,可以在GatewayVirtualService中的host字段使用通配符*

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"  #指定通配符,不限制外部流量的地址
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "*"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /headers
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

问题定位

  1. 检查环境变量INGRESS_HOSTINGRESS_PORT,保证这两个值是有效的 $ kubectl get svc -n istio-system $ echo INGRESS_HOST=$INGRESS_HOST, INGRESS_PORT=$INGRESS_PORT
  2. 校验相同的端口上没有其他istio ingress网格 $ kubectl get gateway --all-namespaces
  3. 校验没有在相同的IP和端口上定义kubernetes ingress资源 $ kubectl get ingress --all-namespaces
  4. 如果没有负载均衡,可以参照上面步骤使用node port方式

卸载

$ kubectl delete gateway httpbin-gateway
$ kubectl delete virtualservice httpbin
$ kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

Ingress(kubernetes)

执行ingress流量控制中的Before you beginBefore you beginDetermining the ingress IP and ports小节的操作,部署httpbin服务。本节介绍如何通过kubernetes的Ingress(非istio的gateway)进行访问。

下面展示如何配置一个80端口的Ingress,用于HTTP流量:

  1. 创建一个istio Gateway,将来自httpbin.example.com:80/status/*的流量分发到service httpbin8000端口 $ kubectl apply -f - <<EOF apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: istio name: ingress spec: rules: - host: httpbin.example.com http: paths: - path: /status/* backend: serviceName: httpbin servicePort: 8000 EOF 注意需要使用 kubernetes.io/ingress.class annotation来告诉istio网关控制器处理该ingress,否则会忽略该ingress。
  2. 使用curl命令访问httpbin服务。Ingress的流量也需要经过istio ingressgateway。 $ curl -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/status/200 HTTP/1.1 200 OK server: istio-envoy date: Fri, 22 May 2020 06:12:56 GMT content-type: text/html; charset=utf-8 access-control-allow-origin: * access-control-allow-credentials: true content-length: 0 x-envoy-upstream-service-time: 20 httpbin服务的发现是通过EDS实现的,使用如下命令查看: $ istioctl proxy-config cluster istio-ingressgateway-569669bb67-b6p5r|grep 8000 httpbin.default.svc.cluster.local 8000 - outbound EDS outbound_.8000_._.httpbin.default.svc.cluster.local - - - EDS
  3. 访问其他未暴露的路径,返回HTTP 404错误: $ curl -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/headers HTTP/1.1 404 Not Found date: Fri, 22 May 2020 06:24:30 GMT server: istio-envoy transfer-encoding: chunked

下一步

TLS

Ingress支持TLS设置。Istio也支持TLS设置,但相关的secret必须存在istio-ingressgateway deployment所在的命名空间中。可以使用cert-manager生成这些证书。

指定路径类型

默认情况下,Istio会将路径视为完全匹配,如果路径使用/**结尾,则该路径为前缀匹配。不支持其他正则匹配。

kubernetes 1.18中新增了一个字段pathType,允许声明为ExactPrefix

指定IngressClass

kubernetes 1.18中新增了一个资源IngressClass,替换了Ingress资源的 kubernetes.io/ingress.class annotation。如果使用该资源,则需要将controller设置为istio.io/ingress-controller

apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress
spec:
  ingressClassName: istio
  ...

卸载

$ kubectl delete ingress ingress
$ kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

安全网关

本节讲述如何使用simple或mutual TLS暴露安全的HTTPS服务。证书是通过SDS进行密钥发现的。

TLS需要的私钥,服务端证书,根证书是使用基于文件装载的方法配置的。

执行ingress流量控制中的Before you beginBefore you beginDetermining the ingress IP and ports小节的操作,部署httpbin服务,并获取 INGRESS_HOSTSECURE_INGRESS_PORT变量。

生成服务端证书和私钥

下面使用openssl生成需要的证书和密钥

  1. 生成一个根证书和一个私钥,用于签名服务的证书 $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt
  2. httpbin.example.com生成一个证书和私钥 $ openssl req -out httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization" $ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in httpbin.example.com.csr -out httpbin.example.com.crt

单主机配置TLS ingress网关

  1. 为ingree网关创建一个secret secret的名字不能以istioprometheus开头,且secret不应该包含token字段 $ kubectl create -n istio-system secret tls httpbin-credential --key=httpbin.example.com.key --cert=httpbin.example.com.crt
  2. server部分定义一个443端口的Gateway,将credentialName指定为httpbin-credential,与secret的名字相同,TLS的mode为SIMPLE。 $ cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: mygateway spec: selector: istio: ingressgateway # use istio default ingress gateway servers: - port: number: 443 name: https protocol: HTTPS tls: mode: SIMPLE credentialName: httpbin-credential # must be the same as secret hosts: - httpbin.example.com EOF
  3. 配置进入Gateway的流量路由。与上一节中的VirtualService相同 $ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: httpbin spec: hosts: - "httpbin.example.com" gateways: - httpbin-gateway http: - match: - uri: prefix: /status - uri: prefix: /delay route: - destination: port: number: 8000 #可以看到tls只是这是在gateway上的,当进入网格之后就不需要了 host: httpbin EOF
  4. 使用curl向SECURE_INGRESS_PORT发送HTTPS请求访问httpbin服务,请求中携带了公钥example.com.crt--resolve标记可以在使用curl访问TLS的网关IP时,在SNI中支持httpbin.example.com--cacert选择支持使用生成的证书校验服务。 -HHost:httpbin.example.com 选项仅在SECURE_INGRESS_PORT不同于实际的网关端口(443)时才会需要,例如,通过映射的NodePort方式访问服务时。 通过将请求发送到/status/418 URL路径时,可以看到httpbin确实被访问了,httpbin服务会返回418 I’m a Teapot代码。. $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" > --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" * Added httpbin.example.com:31967:172.20.127.211 to DNS cache * About to connect() to httpbin.example.com port 31967 (#0) * Trying 172.20.127.211... * Connected to httpbin.example.com (172.20.127.211) port 31967 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: example.com.crt CApath: none * SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 * Server certificate: * subject: O=httpbin organization,CN=httpbin.example.com * start date: May 22 09:03:01 2020 GMT * expire date: May 22 09:03:01 2021 GMT * common name: httpbin.example.com * issuer: CN=example.com,O=example Inc. > GET /status/418 HTTP/1.1 > User-Agent: curl/7.29.0 > Accept: */* > Host:httpbin.example.com > < HTTP/1.1 418 Unknown < server: istio-envoy < date: Fri, 22 May 2020 09:08:29 GMT < x-more-info: http://tools.ietf.org/html/rfc2324 < access-control-allow-origin: * < access-control-allow-credentials: true < content-length: 135 < x-envoy-upstream-service-time: 2 < -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""` * Connection #0 to host httpbin.example.com left intact 查看curl输出中的Server certificate中的信息,上述返回值的最后有一个茶壶的图片,说明运行成功。
  5. 删除老的网关secret,创建一个新的secret,并使用该secret修改ingress网关的凭据 $ kubectl -n istio-system delete secret httpbin-credential $ mkdir new_certificates $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout new_certificates/example.com.key -out new_certificates/example.com.crt $ openssl req -out new_certificates/httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout new_certificates/httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization" $ openssl x509 -req -days 365 -CA new_certificates/example.com.crt -CAkey new_certificates/example.com.key -set_serial 0 -in new_certificates/httpbin.example.com.csr -out new_certificates/httpbin.example.com.crt $ kubectl create -n istio-system secret tls httpbin-credential \ --key=new_certificates/httpbin.example.com.key \ --cert=new_certificates/httpbin.example.com.crt
  6. 使用新的证书链访问httpbin服务 $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert new_certificates/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
  7. 如果使用老的证书访问,则返回错误 $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ > --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" * Added httpbin.example.com:31967:172.20.127.211 to DNS cache * About to connect() to httpbin.example.com port 31967 (#0) * Trying 172.20.127.211... * Connected to httpbin.example.com (172.20.127.211) port 31967 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: example.com.crt CApath: none * Server certificate: * subject: O=httpbin organization,CN=httpbin.example.com * start date: May 22 09:24:07 2020 GMT * expire date: May 22 09:24:07 2021 GMT * common name: httpbin.example.com * issuer: CN=example.com,O=example Inc. * NSS error -8182 (SEC_ERROR_BAD_SIGNATURE) * Peer's certificate has an invalid signature. * Closing connection 0 curl: (60) Peer's certificate has an invalid signature.

多主机配置TLS ingress网关

本节会为多个主机(httpbin.example.comhelloworld-v1.example.com)配置一个ingress网关。ingress网关会在credentialName中查找唯一的凭据。

  1. 删除之前创建的secret并为httpbin重建凭据 $ kubectl -n istio-system delete secret httpbin-credential $ kubectl create -n istio-system secret tls httpbin-credential \ --key=httpbin.example.com.key \ --cert=httpbin.example.com.crt
  2. 启动helloworld-v1 $ cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Service metadata: name: helloworld-v1 labels: app: helloworld-v1 spec: ports: - name: http port: 5000 selector: app: helloworld-v1 --- apiVersion: apps/v1 kind: Deployment metadata: name: helloworld-v1 spec: replicas: 1 selector: matchLabels: app: helloworld-v1 version: v1 template: metadata: labels: app: helloworld-v1 version: v1 spec: containers: - name: helloworld image: istio/examples-helloworld-v1 resources: requests: cpu: "100m" imagePullPolicy: IfNotPresent #Always ports: - containerPort: 5000 EOF
  3. helloworld-v1.example.com创建证书和私钥 $ openssl req -out helloworld-v1.example.com.csr -newkey rsa:2048 -nodes -keyout helloworld-v1.example.com.key -subj "/CN=helloworld-v1.example.com/O=helloworld organization" $ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 1 -in helloworld-v1.example.com.csr -out helloworld-v1.example.com.crt
  4. helloworld-credential创建secret $ kubectl create -n istio-system secret tls helloworld-credential --key=helloworld-v1.example.com.key --cert=helloworld-v1.example.com.crt
  5. 定义两个网关,网关端口为443。在credentialName字段分别设置httpbin-credentialhelloworld-credential,TLS模式为SIMPLE。 $ cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: mygateway spec: selector: istio: ingressgateway # use istio default ingress gateway servers: - port: number: 443 name: https-httpbin #httpbin的gateway配置 protocol: HTTPS tls: mode: SIMPLE credentialName: httpbin-credential hosts: - httpbin.example.com - port: number: 443 name: https-helloworld #https-helloword的gateway配置 protocol: HTTPS tls: mode: SIMPLE credentialName: helloworld-credential hosts: - helloworld-v1.example.com EOF
  6. 配置gateway的流量路由,为新应用添加对应的virtual service $ cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: helloworld-v1 spec: hosts: - helloworld-v1.example.com gateways: - mygateway http: - match: - uri: exact: /hello route: - destination: host: helloworld-v1 port: number: 5000 EOF
  7. helloworld-v1.example.com发送HTTPS请求 $ curl -v -HHost:helloworld-v1.example.com --resolve "helloworld-v1.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example.com.crt "https://helloworld-v1.example.com:$SECURE_INGRESS_PORT/hello" ... < HTTP/1.1 200 OK < content-type: text/html; charset=utf-8 < content-length: 60 < server: istio-envoy < date: Sat, 23 May 2020 07:38:40 GMT < x-envoy-upstream-service-time: 143 < Hello version: v1, instance: helloworld-v1-5dfcf5d5cd-2l44c * Connection #0 to host helloworld-v1.example.com left intact
  8. httpbin.example.com发送请求 $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""` * Connection #0 to host httpbin.example.com left intact

配置一个mutual TLS ingress网关

删除之前的secreting创建一个新的secret,server会使用该CA证书校验client,使用cacert保存CA证书。

$ kubectl -n istio-system delete secret httpbin-credential
$ kubectl create -n istio-system secret generic httpbin-credential --from-file=tls.key=httpbin.example.com.key \
--from-file=tls.crt=httpbin.example.com.crt --from-file=ca.crt=example.com.crt
  1. 将gateway的TLS模式设置为MUTUAL $ cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: mygateway spec: selector: istio: ingressgateway # use istio default ingress gateway servers: - port: number: 443 name: https protocol: HTTPS tls: mode: MUTUAL #TLS模式设置为MUTUAL credentialName: httpbin-credential # must be the same as secret hosts: - httpbin.example.com EOF
  2. 使用先前的方式发送HTTPS请求,可以看到访问失败 $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ > --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" * Added httpbin.example.com:31967:172.20.127.211 to DNS cache * About to connect() to httpbin.example.com port 31967 (#0) * Trying 172.20.127.211... * Connected to httpbin.example.com (172.20.127.211) port 31967 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * CAfile: example.com.crt CApath: none * NSS: client certificate not found (nickname not specified) * NSS error -12227 (SSL_ERROR_HANDSHAKE_FAILURE_ALERT) * SSL peer was unable to negotiate an acceptable set of security parameters. * Closing connection 0 curl: (35) NSS: client certificate not found (nickname not specified)
  3. 生成client的证书和私钥。在curl中传入客户端的证书和私钥,使用--cert传入客户端证书,使用--key传入私钥 $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" --cacert example.com.crt --cert ./client.example.com.crt --key ./client.example.com.key "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418" ... -=[ teapot ]=- _...._ .' _ _ `. | ."` ^ `". _, \_;`"---"`|// | ;/ \_ _/ `"""` * Connection #0 to host httpbin.example.com left intact

istio支持几种不同的Secret格式,来支持与多种工具的集成,如cert-manager:

  • 一个TLS Secret使用tls.keytls.crt;对于mutual TLS,会用到ca.crt
  • 一个generic Secret会用到keycert;对于mutual TLS,会用到cacert
  • 一个generic Secret会用到keycert;对于mutual TLS,会用到一个单独的名为 <secret>-cacert的generic Secret,以及一个cacert key。例如httpbin-credential 包含keycerthttpbin-credential-cacert 包含cacert

问题定位

  • 检查INGRESS_HOSTSECURE_INGRESS_PORT环境变量 $ kubectl get svc -n istio-system $ echo INGRESS_HOST=$INGRESS_HOST, SECURE_INGRESS_PORT=$SECURE_INGRESS_PORT
  • 检查istio-ingressgateway控制器的错误日志 $ kubectl logs -n istio-system "$(kubectl get pod -l istio=ingressgateway \ -n istio-system -o jsonpath='{.items[0].metadata.name}')"
  • 校验istio-system命名空间中成功创建了secret。上例中应该存在httpbin-credentialhelloworld-credential $ kubectl -n istio-system get secrets
  • 校验ingress网关agent将密钥/证书对上传到了ingress网关 $ kubectl logs -n istio-system "$(kubectl get pod -l istio=ingressgateway \ -n istio-system -o jsonpath='{.items[0].metadata.name}')"

定位mutul TLS问题

  • 校验CA加载到了 istio-ingressgateway pod中,查看是否存在example.com.crt $ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-ca-certs
  • 如果创建了istio-ingressgateway-ca-certs secret,但没有加载CA证书,删除ingress网关pod,强制加载该证书 $ kubectl delete pod -n istio-system -l istio=ingressgateway
  • 校验CA证书的Subject字段是否正确 $ kubectl exec -i -n istio-system $(kubectl get pod -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -- cat /etc/istio/ingressgateway-ca-certs/example.com.crt | openssl x509 -text -noout | grep 'Subject:' Subject: O=example Inc., CN=example.com log中可以看到添加了httpbin-credential secret。如果使用mutual TLS,那么也会出现 httpbin-credential-cacert secret。校验log中显示了网关agent从ingress网关接收到了SDS请求,资源的名称为 httpbin-credential,且ingress网关获取到了密钥/证书对。如果使用了mutual TLS,日志应该显示将密钥/证书发送到ingress网关,网关agent接收到了带 httpbin-credential-cacert 资源名称的SDS请求,并回去到了根证书。

卸载

  1. 删除Gateway配置,VirtualService和secret $ kubectl delete gateway mygateway $ kubectl delete virtualservice httpbin $ kubectl delete --ignore-not-found=true -n istio-system secret httpbin-credential \ helloworld-credential $ kubectl delete --ignore-not-found=true virtualservice helloworld-v1
  2. 删除证书目录 $ rm -rf example.com.crt example.com.key httpbin.example.com.crt httpbin.example.com.key httpbin.example.com.csr helloworld-v1.example.com.crt helloworld-v1.example.com.key helloworld-v1.example.com.csr client.example.com.crt client.example.com.csr client.example.com.key ./new_certificates
  3. 停止httpbinhelloworld-v1 服务: $ kubectl delete deployment --ignore-not-found=true httpbin helloworld-v1 $ kubectl delete service --ignore-not-found=true httpbin helloworld-v1

不终止TLS的ingress网关

上一节中描述了如何配置HTTPS ingree来访问一个HTTP服务。本节中描述如何配置HTTPS ingrss来访问HTTPS服务等。通过配置ingress网关来执行SNI方式的访问,而不会在请求进入ingress时终止TLS。

本例中使用一个NGINX服务器作为HTTPS服务。

生成客户端和服务端的证书和密钥

  1. 生成一个根证书和私钥,用于签名服务 $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt
  2. nginx.example.com创建证书和私钥 $ openssl req -out nginx.example.com.csr -newkey rsa:2048 -nodes -keyout nginx.example.com.key -subj "/CN=nginx.example.com/O=some organization" $ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in nginx.example.com.csr -out nginx.example.com.crt

部署NGINX服务

  1. 创建kubernetes Secret保存服务的证书 $ kubectl create secret tls nginx-server-certs --key nginx.example.com.key --cert nginx.example.com.crt
  2. 为NGINX服务创建配置文件 $ cat <<EOF > ./nginx.conf events { } http { log_format main '$remote_addr - $remote_user [$time_local] $status ' '"$request" $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log /var/log/nginx/access.log main; error_log /var/log/nginx/error.log; server { listen 443 ssl; root /usr/share/nginx/html; index index.html; server_name nginx.example.com; ssl_certificate /etc/nginx-server-certs/tls.crt; ssl_certificate_key /etc/nginx-server-certs/tls.key; } } EOF
  3. 为NGINX服务创建kubernetes configmap $ kubectl create configmap nginx-configmap --from-file=nginx.conf=./nginx.conf
  4. 部署NGINX服务 $ cat <<EOF | istioctl kube-inject -f - | kubectl apply -f - apiVersion: v1 kind: Service metadata: name: my-nginx labels: run: my-nginx spec: ports: - port: 443 protocol: TCP selector: run: my-nginx --- apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx spec: selector: matchLabels: run: my-nginx replicas: 1 template: metadata: labels: run: my-nginx spec: containers: - name: my-nginx image: nginx ports: - containerPort: 443 volumeMounts: - name: nginx-config mountPath: /etc/nginx readOnly: true - name: nginx-server-certs mountPath: /etc/nginx-server-certs readOnly: true volumes: - name: nginx-config configMap: name: nginx-configmap - name: nginx-server-certs secret: secretName: nginx-server-certs #保存了NGINX服务的证书和私钥 EOF
  5. 为了测试NGINX服务部署成功,向服务发送不使用证书的方式请求,并校验打印信息是否正确: $ kubectl exec -it $(kubectl get pod -l run=my-nginx -o jsonpath={.items..metadata.name}) -c istio-proxy -- curl -v -k --resolve ... * Server certificate: * subject: CN=nginx.example.com; O=some organization * start date: May 25 02:09:02 2020 GMT * expire date: May 25 02:09:02 2021 GMT * issuer: O=example Inc.; CN=example.com * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. ...

配置一个ingress gateway

  1. 定义一个gateway,端口为443.注意TLS的模式为PASSTHROUGH,表示gateway会放行ingress流量,不会终止TLS $ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: mygateway spec: selector: istio: ingressgateway # use istio default ingress gateway servers: - port: number: 443 name: https protocol: HTTPS tls: mode: PASSTHROUGH #不终止TLS hosts: - nginx.example.com EOF
  2. 配置经过Gateway的流量路由 $ kubectl apply -f - <<EOF apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: nginx spec: hosts: - nginx.example.com gateways: - mygateway tls: - match: - port: 443 #将gateway的流量导入kubernetes的my-nginx service sniHosts: - nginx.example.com route: - destination: host: my-nginx port: number: 443 EOF
  3. 根据指导配置SECURE_INGRESS_PORTINGRESS_HOST环境变量
  4. 通过ingress访问nginx,可以看到访问成功 $ curl -v --resolve nginx.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST --cacert example.com.crt https://nginx.example.com:$SECURE_INGRESS_PORT ... * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 * Server certificate: * subject: O=some organization,CN=nginx.example.com * start date: May 25 02:09:02 2020 GMT * expire date: May 25 02:09:02 2021 GMT * common name: nginx.example.com * issuer: CN=example.com,O=example Inc. ... <title>Welcome to nginx!</title> ...

卸载

  1. 移除kubernetes资源 $ kubectl delete secret nginx-server-certs $ kubectl delete configmap nginx-configmap $ kubectl delete service my-nginx $ kubectl delete deployment my-nginx $ kubectl delete gateway mygateway $ kubectl delete virtualservice nginx
  2. 删除证书和密钥 $ rm example.com.crt example.com.key nginx.example.com.crt nginx.example.com.key nginx.example.com.csr
  3. 删除生成的配置文件 $ rm ./nginx.conf

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • 验证Prometheus alertmanager邮件发送

    新环境上配置alertmanager时出现了“Client was not authenticated to send anonymous mail durin...

    charlieroro
  • 负载均衡策略导致后端程序访问异常

      最近新部署了openshift集群,由于使用了自签证书,浏览器访问集群的https服务会报告警(如下图),在使用浏览器(特别是IE)访问openshift ...

    charlieroro
  • docker网络之macvlan

    macvlan接口类型简单说类似于子接口,但相比子接口来说,macvlan接口拥有自己独立的mac地址,因此使用macvlan接口可以允许更多的二层操作。mac...

    charlieroro
  • 使用dig执行手动DNS查询

    dig 是一个命令行DNS查询实用程序,允许您诊断域名解析问题。如果您没有域名,建议您先去这里注册一个域名,将域名解析到您的服务器,您可以使用腾讯云云解析进行快...

    穿鞋跑得快
  • PHPer友好|可能是最简单易懂的机器学习入门资料

    前段时间笔者推送了一条 google 官方机器学习速成课程的链接(https://developers.google.com/machine-learn...

    猿哥
  • 《Prometheus监控实战》第5章 服务发现

    yeedomliu
  • 一个DNS数据包的惊险之旅

    “小子,快去查一下www.paypal.com的IP地址,我急用,晚了我弄你!”,暴躁老哥一把关上了门,留我一个DNS数据包在冷冰冰的房间。

    轩辕之风
  • Redhat apache基本配置修改http.conf

    ServerTokens Prod #OS改为Prod屏蔽系统版本号显示 DocumentRoot “/var/www/html” #这个是根目录 # Al...

    苦咖啡
  • 如何帅(zhuang)气(x)的发邮件

    数据header部分定义协议版本,内容类型,与传输编码。发送人,收件人,抄送人,主题和正文,最后是数据结束符号。

    江湖安得便相忘
  • 最新数据科学和机器学习 MOOC 资源,成为高手的个性化课程组合

    【新智元导读】今天我们要介绍的主人公叫 David Venturi。一年前他还没有编程背景,凭着对数学的爱好开始上网自学。后来他被加拿大一所大学的计算机科学专业...

    新智元

扫码关注云+社区

领取腾讯云代金券