专栏首页charlierorominikube配置CRI-O作为runtime并指定flannel插件

minikube配置CRI-O作为runtime并指定flannel插件

  使用crio作为runtime后,容器的启动将不依赖docker相关的组件,容器进程更加简洁。如下使用crio作为runtime启动一个nginx的进程信息如下:根进程(1)->conmon->nginx。conmon作用于crio和runc(OCI实现)之间,用于在crio启动容器后托管容器,更多参见conmon

root     15586     1  0 16:49 ?        00:00:00 /usr/local/bin/conmon --syslog -c a4f089f6b251c6269e2f79c41cec0317f4a65729b6075c77bbf4337206050501 -n k8s_nginx-test_nginx-test-24cjg_default_55bbcfe7-d63c-468b-bbcc-35a8b6c71eb9
root     15609 15586  0 16:49 ?        00:00:00 nginx: master process nginx -g daemon off;

安装minikube

安装cri-o(以下步骤来自官方文档):

  • 安装依赖库
yum install -y \
  btrfs-progs-devel \
  containers-common \
  device-mapper-devel \
  git \
  glib2-devel \
  glibc-devel \
  glibc-static \
  go \
  gpgme-devel \
  libassuan-devel \
  libgpg-error-devel \
  libseccomp-devel \
  libselinux-devel \
  pkgconfig \
  runc
  • 编译CRI-O,在编译CRI-O时可以指定Build-tag。当前的CRI-O需要golang 12.x版本来编译
git clone https://github.com/cri-o/cri-o # or your fork
cd cri-o
make
sudo make install
  • 编译Conmon
git clone https://github.com/containers/conmon
cd conmon
make
sudo make install

crio的配置文件默认为/etc/crio/crio.conf,可以通过命令crio config --default > /etc/crio/crio.conf来生成默认配置文件。

设置CNI网络(以下步骤来自官方文档)

git clone https://github.com/containernetworking/plugins
cd plugins
git checkout v0.8.1
./build_linux.sh # or build_windows.sh
sudo mkdir -p /opt/cni/bin
sudo cp bin/* /opt/cni/bin/
  • 编译完CNI之后,把/opt/cni/bin中的二进制文件拷贝到/etc/crio/crio.conf的crio.network.plugin_dir目录,默认为/usr/libexec/cni;并将cni配置放到crio.network.network_dir目录

启动CRI-O

  • 在cri-o的源码目录下执行如下步骤,启动CRI-O
sudo make install.systemd
sudo systemctl daemon-reload
sudo systemctl enable crio
sudo systemctl start crio

使用crio-status命令

  • 使用crio-status config可以查看当前crio的配置

安装CRI-O命令行工具crictl

  • crictl用法与docker命令类似,可以参见官方文档
# go get github.com/kubernetes-sigs/cri-tools/cmd/crictl
# cp /root/go/bin/crictl /usr/local/bin
# crictl --runtime-endpoint unix:///var/run/crio/crio.sock version
Version:  0.1.0
RuntimeName:  cri-o
RuntimeVersion:  1.15.1-dev
RuntimeApiVersion:  v1alpha1
  • crictl默认会读取/etc/crictl.yaml中的runtime-endpoint配置
# cat /etc/crictl.yaml
runtime-endpoint: unix:///var/run/crio/crio.sock
image-endpoint: unix:///var/run/crio/crio.sock

启动minikube并配置使用CRI-O

minikube start --container-runtime=cri-o --vm-driver=none

crictl简单使用

在启动minikube之后会启动相关组件pod,使用circtl ps可以看到相关的容器信息,最后一列为POD ID。更多参见crictl

[root@iZj6cid8uez7g44i1t0k7tZ net.d]# crictl ps
CONTAINER           IMAGE                                                                                                             CREATED             STATE               NAME                      ATTEMPT             POD ID
b69e8be1ef2b0       gcr.io/k8s-minikube/storage-provisioner@sha256:088daa9fcbccf04c3f415d77d5a6360d2803922190b675cb7fc88a9d2d91985a   About an hour ago   Running             storage-provisioner       0                   282d5beebf847
dd57045952649       bf261d157914477ee1a5969d28ec687f3fbfc9fb5a664b22df78e57023b0e03b                                                  About an hour ago   Running             coredns                   0                   4f7a8f3cac5c4
a9df5247ede0f       bf261d157914477ee1a5969d28ec687f3fbfc9fb5a664b22df78e57023b0e03b                                                  About an hour ago   Running             coredns                   0                   6448effa2f7cd
dc1027c8d94c5       c21b0c7400f988db4777858edd13b6d3930d62d7ccf026d2415485a52037f384                                                  About an hour ago   Running             kube-proxy                0                   0436f736f2a4a
25cb103bc2e1e       k8s.gcr.io/kube-addon-manager@sha256:3e315022a842d782a28e729720f21091dde21f1efea28868d65ec595ad871616             About an hour ago   Running             kube-addon-manager        0                   85ceee77c5c70
cf7378a82993d       301ddc62b80b16315d3c2653cf3888370394277afb3187614cfa20edc352ca0a                                                  About an hour ago   Running             kube-scheduler            0                   baf3c10a81831
60d9bcf7a4b83       06a629a7e51cdcc81a5ed6a3e6650348312f20c954ac52ee489a023628ec9c7d                                                  About an hour ago   Running             kube-controller-manager   0                   877a92f202a5f
7a67b324cd8c7       b2756210eeabf84f3221da9959e9483f3919dc2aaab4cd45e7cd072fcbde27ed                                                  About an hour ago   Running             etcd                      0                   74fe384e1645b
355ba11ac783f       b305571ca60a5a7818bda47da122683d75e8a1907475681ee8b1efbd06bff12e                                                  About an hour ago   Running             kube-apiserver            0                   d112f1dc64113

使用crictl inspect CONTAINER_ID可以查看容器的详细信息,使用circtl inspectp POD_ID查看pod的详细信息。crictl inspect CONTAINER_ID|grep sandboxId出来的值就是该容器对应的pod的POD_ID。

配置使用flannel插件

使用如下方式启动minikube

minikube start \
    --extra-config=controller-manager.allocate-node-cidrs=true \
    --extra-config=controller-manager.cluster-cidr=10.233.64.0/18 \
    --extra-config=kubelet.network-plugin=cni \
    --extra-config=kubelet.pod-cidr=10.233.64.0/18 \
    --network-plugin=cni \
    --container-runtime=cri-o \
    --vm-driver=none

按照官方命令安装flannel插件,并查看coredns是否正常启动(cni启动之前coredns处于pending状态)。执行下面命令前需要确保/etc/cni/net.d/为空(flannel自动生成)或配置文件正确,否则会有错误

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml

如果coredns出现如下错误,说明/etc/cni/net.d/中的配置文件版本字段错误,可以参考flannel官方配置,将cniVersion字段修改为"0.3.1",这样coredns稍后会正常启动。使用crictl inspectp POD_ID可以看到network设置为minikube启动参数--extra-config=kubelet.pod-cidr指定的值

cannot convert version ["" "0.1.0" "0.2.0"] to 0.4.0

正常启动后可以在/run/flannel/subnet.env中看到配置的flannel信息,

# cat /run/flannel/subnet.env
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.233.64.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true

查看本地接口,可以看到flannel接口创建成功,后续新创建的pod将使用flannel网络

]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:16:3e:04:eb:0e brd ff:ff:ff:ff:ff:ff
3: mybridge: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 0a:be:69:1e:02:70 brd ff:ff:ff:ff:ff:ff
7: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/ether 06:66:cd:4f:d2:9a brd ff:ff:ff:ff:ff:ff
8: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 02:02:85:cf:25:dd brd ff:ff:ff:ff:ff:ff
301: veth1b2b30e0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP mode DEFAULT group default
    link/ether 8e:47:4b:b8:10:be brd ff:ff:ff:ff:ff:ff link-netnsid 0
302: veth2147d829@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP mode DEFAULT group default
    link/ether fa:3f:fe:5d:91:82 brd ff:ff:ff:ff:ff:ff link-netnsid 1
303: veth54baeef4@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP mode DEFAULT group default
    link/ether 9a:99:0f:82:ff:34 brd ff:ff:ff:ff:ff:ff link-netnsid 2

flannel原理如下

flannel支持Vxlan,Host-gw和UDP模式,其中UDP一般用于debug模式,可在kube-system命名空间的configmap kube-flannel-cfg中查看flannel的运行模式。更多参见官方文档

TIPS:

  • crio主要有如下4个配置文件

File

Description

crio.conf(5)

CRI-O Configuration file

policy.json(5)

Signature Verification Policy File(s)

registries.conf(5)

Registries Configuration file

storage.conf(5)

Storage Configuration file

  • minikube启动出现:sudo: crictl: command not found,解决办法:将crictl放到/usr/bin目录,参考该issue。原因是/etc/sudoers的secure_path中没有该路径
  • minikube启动出现:[certs] certificate apiserver-kubelet-client not signed by CA certificate ca: crypto/rsa: verification error,解决办法:rm /var/lib/minikube/certs,参考该issue

参考:

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • docker网络之bridge

    https://blog.csdn.net/u014027051/article/details/53908878/

    charlieroro
  • 基于eBPF的微服务网络安全(Cilium 1)

    翻译自:Network security for microservices with eBPF

    charlieroro
  • docker cgroup技术之cpu和cpuset

      在centos7的/sys/fs/cgroup下面可以看到与cpu相关的有cpu,cpuacct和cpuset 3个subsystem。cpu用于对cpu使...

    charlieroro
  • 行业分析|《奇葩说》马东告诉你内容经济的本质

    米未传媒CEO马东老师在混沌大学上分享了其对内容经济的深刻理解,同时总结了头部内容《奇葩说》运营创新的方法论,因为分享采用以访谈形式,结构略显混乱,笔者尝试梳理...

    用户2559057
  • Webpack中hash与chunkhash的区别,以及js与css的hash指纹解耦方案

    文件的hash指纹通常作为前端静态资源实现增量更新的方案之一,Webpack是目前最流行的开源编译工具之一,其强大的功能也带来很多坑(当然,大部分麻烦其实都可以...

    寒月十八
  • [三]java8 函数式编程Stream 概念深入理解 Stream 运行原理 Stream设计思路

            流不是存储元素的数据结构;相反,它通过一个计算操作的管道,从一个数据源,如数据结构、数组、生成器函数或i/o通道中传递元素

    noteless
  • 论文阅读:《Improving Content-based and Hybrid Music Recommendation using Deep Learning》

    https://blog.csdn.net/u011239443/article/details/79984751

    用户1621453
  • Tomcat集群部署

    用户1456517
  • 共享单车火爆的背后思考:是不是真的解决了“最后一米”?

    镁客网
  • 利用Nginx反向代理绕过腾讯云黑洞隔离

    我这里备用机用的是windows phpstudy nginx 环境,其他环境请自行测试!

    墨渊

扫码关注云+社区

领取腾讯云代金券