docker 套娃在CI中应用解析

docker in docker 简介

docker里嵌套运行docker，本文讲解其在jenkins和gitlab-runner 中的调用流程

一、用于jenkins

容器化部署jenkins时调用docker命令集成CI功能

1. [root@ops-demo~]# docker inspect jenkins --format="{{json .Mounts}}"
2. [{"Type":"bind","Source":"/usr/bin/docker","Destination":"/usr/bin/docker","Mode":"","RW":true,"Propagation":"rprivate"},{"Type":"volume","Name":"jenkins","Source":"/var/lib/docker/volumes/jenkins/_data","Destination":"/var/jenkins_home","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"bind","Source":"/var/run/docker.sock","Destination":"/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"}]

调用流程

执行如下命令可以发现jenkins是使用挂载进宿主机的docker.sock 来调用的docker服务，即：在jenkins容器内和宿主机上执行docker命令，效果是一样的，操作的结果，容器内外查看效果一样，要是构建的时候在jenkins容器执行 docker stop jenkins 会发生什么？

1. [root@ops-demo ~]# docker images | wc -l
2. 124
3. [root@dotnet ~]# docker exec -it jenkins bash
4. jenkins@855107e4687c:/$ docker images | wc -l
5. 124
6. jenkins@855107e4687c:/$
8. # 容器内外的镜像数据一样，说明用的是用一个docker服务

二、用于gitlab-runner

gitlab-runner可以用容器方式和非容器方式运行，本文讲docker套娃，于是采用容器方式运行gitlab-runner。

gitlab-runner的executor有多种，对于docker套娃有两种方式，下文讲第二种调用流程。

• 1、像上文jenkins一样，直接挂载 /var/run/docker.sock
• 2、通过在gitlab-runner里新建一个完整的docker服务，这个完整的服务docker官方有提供一个镜像: docker:dind

docker套娃调用流程解析

注意看中文注释

gitlab-runner config.toml 配置样例

1. [[runners]]
2. url = "https://gitlab.com/"
3. token = TOKEN
4. executor = "docker" #docker 执行器
5. [runners.docker]
6. tls_verify = false
7. image = "docker:24.0.5" # 用于gitalb-runner容器里提供docker client
8. privileged = true
9. disable_cache = false
10. volumes = ["/cache"]
11. [runners.cache]
12. [runners.cache.s3]
13. [runners.cache.gcs]

gitlab Project中的 gitlab-ci.yaml 配置样例

1. default:
2. image: docker:24.0.5 #用于提供 docker client
3. services:
4. - docker:24.0.5-dind # 用于启动 Docker daemon
5. before_script:
6. - docker info
7. variables:
8. # When using dind service, you must instruct docker to talk with the
9. # daemon started inside of the service. The daemon is available with
10. # a network connection instead of the default /var/run/docker.sock socket. # 注意这句提示
11. #
12. # The 'docker' hostname is the alias of the service container as described at
13. # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
14. #
15. # If you're using GitLab Runner 12.7 or earlier with the Kubernetes executor and Kubernetes 1.6 or earlier,
16. # the variable must be set to tcp://localhost:2375 because of how the
17. # Kubernetes executor connects services to the job container
18. # DOCKER_HOST: tcp://localhost:2375
19. #
20. DOCKER_HOST: tcp://docker:2376 #dind启动的Docker daemon所监听的tcp端口
21. #
22. # This instructs Docker not to start over TLS.
23. DOCKER_TLS_CERTDIR: ""
24. build:
25. stage: build
26. script:
27. - docker build -t my-docker-image .
28. - docker run my-docker-image /script/to/run/tests

启动docker:24.0.5看看

1. [root@ops-demo ~]# docker run -it --privileged --name docker-client docker:24.0.5 sh/
2. # docker pserror during connect: Get "http://docker:2375/v1.24/containers/json": dial tcp: lookup docker on xxx: no such host

可以看到默认请求的host是 docker ，但是端口为什么不是2376呢？再看

1. [root@ops-demo ~]# docker run -it --privileged --name docker-client docker:24.0.5 sh
2. / # env
3. HOSTNAME=c3b5946753f1
4. SHLVL=3
5. HOME=/root
6. DIND_COMMIT=d58df1fc6c866447ce2cd129af10e5b507705624
7. TERM=xterm
8. PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
9. DOCKER_VERSION=24.0.5
10. DOCKER_TLS_CERTDIR=/certs
11. DOCKER_HOST=tcp://docker:2375 # 这里通过环境变量设置的，上文gitlab-ci.yaml中自定义为dind默认监听的端口2376
12. DOCKER_BUILDX_VERSION=0.11.2
13. DOCKER_COMPOSE_VERSION=2.21.0
14. PWD=/
15. / #

启动dind验证下监听的是不是2376端口？

time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

1. [root@ops-demo~]# docker run --privileged --name dind docker:dind
2. ...
3. time="2024-05-29T10:16:08.938258364Z" level=info msg="Docker daemon" commit=8e96db1 containerd-snapshotter=false storage-driver=overlay2 version=26.1.3
4. time="2024-05-29T10:16:08.939055975Z" level=info msg="Daemon has completed initialization"
5. time="2024-05-29T10:16:09.047817847Z" level=info msg="API listen on /var/run/docker.sock"
6. time="2024-05-29T10:16:09.048087250Z" level=info msg="API listen on [::]:2376"

调用流程

经过以上分析，可以捋出gitlab-runner docker套娃的工作流程

• 1、宿主机docker启动gitlab-runner容器
• 2、gitlab-runner调用 docker:24.0.5提供的docker client执行各类docker命令，包括启动docker:dind容器
• 3、然后docker client调用的Docker daemon是docker:dind提供的

gitlab-runner的工作流程

给出一张gitlab-runner的工作流程图方便理解