Gitlab和gitlab运行程序使用坞内对接器，无法注册新层(“未能创建btrfs快照:设备不合适的ioctl”)
Gitlab和gitlab运行程序使用坞内对接器，无法注册新层(“未能创建btrfs快照:设备不合适的ioctl”)
提问于 2022-10-11 12:06:18
我在使用docker build从.gitlab-ci.yml文件中运行docker:dind时遇到了问题，只有在添加新层时才会发生这种情况。
我的gitlab和gitlab-runner都是作为容器运行在synology NAS上的顶级Docker中(它使用btrfs作为存储驱动程序)，如下所示：
version: '3.6'
services:
gitlab:
container_name: gitlab
image: 'gitlab/gitlab-ce:latest'
restart: unless-stopped
hostname: 'git.MY_FQDN_REDACTED.com'
environment:
VIRTUAL_HOST: git.MY_FQDN_REDACTED.com
VIRTUAL_PORT: 80
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://git.MY_FQDN_REDACTED.com'
nginx['listen_port'] = 80
nginx['listen_https'] = false
gitlab_rails['gitlab_shell_ssh_port'] = 2222
registry_external_url 'https://git.MY_FQDN_REDACTED.com:5112'
registry_nginx['ssl_certificate'] = "/certs/MY_FQDN_REDACTED.com.crt"
registry_nginx['ssl_certificate_key'] = "/certs/MY_FQDN_REDACTED.com.key"
expose:
- 80
ports:
- "2222:22"
- "5112:5112"
networks:
- default
- proxy
volumes:
- 'gitlab-config:/etc/gitlab'
- 'gitlab-logs:/var/log/gitlab'
- 'gitlab-data:/var/opt/gitlab'
- ${SSL_DIR}:/certs
shm_size: '256m'
gitlab-build-runner:
container_name: gitlab-build-runner
image: 'gitlab/gitlab-runner:latest'
networks:
- default
environment:
GIT_SSL_NO_VERIFY: "true"
CA_CERTIFICATES_PATH: "/ca_certs"
privileged: true
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- './gitlab-build-runner/config/config.toml:/etc/gitlab-runner/config.toml'
- '/volume1/docker/ents/ssl/MY_CA_INTERMEDIATE_CERT.pem:/ca_certs/ca.pem'
restart: unless-stopped
networks:
default:
proxy:
external: true
name: proxy
volumes:
gitlab-logs:
gitlab-config:
gitlab-data:gitlab接口位于后面，通过nginx反向代理(配置未显示)进行访问，直接访问gitlab容器注册表(端口5112)并终止自己的TLS。正如您所看到的，gitlab-runner可以访问父端口套接字，因此可以在顶层生成运行程序实例(在这个级别上，不是在坞中)。
Gitlab配置(/etc/gitlab-runner/config.toml)如下：
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "b66dc462d510"
url = "http://git.MY_FQDN_REDACTED.com/"
id = 6
token = "REDACTED"
token_obtained_at = 2022-10-10T21:57:00Z
token_expires_at = 0001-01-01T00:00:00Z
executor = "docker"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker:stable"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache", "/volume1/docker/ents/ssl/MY_CA_INTERMEDIATE_CERT.pem:/ca_certs/ca.pem"]
shm_size = 0
pre_build_script = """
apk update >/dev/null
apk add ca-certificates > /dev/null
rm -rf /var/cache/apk/*
cp /ca_certs/ca.pem /usr/local/share/ca-certificates/ca.crt
update-ca-certificates --fresh > /dev/null
"""我有一个带有Dockerfile的基本项目，它使用.gitlab-ci.yml，如下所示：
variables:
DOCKER_TLS_CERTDIR: ""
DOCKER_HOST: tcp://docker:2375
GIT_SSL_NO_VERIFY: "true"
docker-build:
# Use the official docker image.
image: docker:dind
stage: build
tags:
- build
services:
- name: docker:dind
command: ["--insecure-registry=git.MY_FQDN_REDACTED.com:5112"]
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
# Default branch leaves tag empty (= latest tag)
# All other branches are tagged with the escaped branch name (commit ref slug)
script:
- |
if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
tag=""
echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
else
tag=":$CI_COMMIT_REF_SLUG"
echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
fi
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
- docker push "$CI_REGISTRY_IMAGE${tag}"
# Run this job in a branch where a Dockerfile exists
rules:
- if: $CI_COMMIT_BRANCH
exists:
- Dockerfile现在，如果我的Dockerfile大部分是空白的，比如：
FROM debian:stable-slim那我的管道就能用了。但是，如果我将图层添加到Dockerfile中，则图像将生成、标记并推送到gitlab的容器注册表，如下所示：
FROM debian:stable-slim
RUN apt update -y然后我的管道与failed to register layer: Failed to create btrfs snapshot: inappropriate ioctl for device失败。
以下是失败时的完整管道输出：
Running with gitlab-runner 15.4.0 (43b2dc3d)
on b66dc462d510 oPaPyVQX
Preparing the "docker" executor
00:33
Using Docker executor with image docker:dind ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:c82a93f89b8dca89e129732754f5fe2e948379bd4d28f117036589ab6d039941 for docker:dind with digest docker@sha256:5da8f946c2b2b9e37b6554680ef3cac95875cb4f5bf66001c80a5e0cc726ddac ...
Waiting for services to be up and running (timeout 30 seconds)...
Using docker image sha256:c82a93f89b8dca89e129732754f5fe2e948379bd4d28f117036589ab6d039941 for docker:dind with digest docker@sha256:5da8f946c2b2b9e37b6554680ef3cac95875cb4f5bf66001c80a5e0cc726ddac ...
Preparing environment
00:06
Running on runner-opapyvqx-project-3-concurrent-0 via f43fcf476734...
Getting source from Git repository
00:05
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/mark/debian-test/.git/
Checking out f49cef34 as main...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:16
Using docker image sha256:c82a93f89b8dca89e129732754f5fe2e948379bd4d28f117036589ab6d039941 for docker:dind with digest docker@sha256:5da8f946c2b2b9e37b6554680ef3cac95875cb4f5bf66001c80a5e0cc726ddac ...
$ docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then # collapsed multi-line command
Running on default branch 'main': tag = 'latest'
$ docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
Step 1/2 : FROM debian:stable-slim
stable-slim: Pulling from library/debian
5c556efcf56e: Pulling fs layer
5c556efcf56e: Download complete
5c556efcf56e: Pull complete
Digest: sha256:92ed72016796475ea1f18f84cd8e2f8519ece3a9ea27fdde1157464078ea5371
Status: Downloaded newer image for debian:stable-slim
---> 6a53e0f5c32d
Step 2/2 : RUN apt update -y
Failed to create btrfs snapshot: inappropriate ioctl for device
ERROR: Job failed: exit code 1知道我是怎么克服的吗？我尝试过在CI作业变量中设置btrfs存储驱动程序，但这不起作用。我遗漏了什么？
编辑:我刚才尝试过的另一件事是通过堆栈匹配坞版本，以便也匹配Btrfs版本，所以.在gitlab-runner config.toml中
[runners.docker]
tls_verify = false
image = "docker:20.10.3"在.gitlab-ci.yml中
docker-build:
image: docker:20.10.3-dind
stage: build
services:
- name: docker:20.10.3-dind现在顶级主机(nas)、gitlab运行实例和构建容器之间的对接版本都匹配20.10.3，但是Btrfs版本仍然不匹配.
顶级主机(nas)：
Server Version: 20.10.3
Storage Driver: btrfs
Build Version: Btrfs v4.0
Library Version: 101
Logging Driver: json-file码头:dind构建容器：
Server Version: 20.10.3
Storage Driver: btrfs
Build Version: Btrfs v4.20.1
Library Version: 102
Logging Driver: json-file不知道还能尝试什么。我绝对不想改变有关synology的btrfs实现的任何内容。
回答 1
Stack Overflow用户
发布于 2022-10-11 15:21:10
好的，所以我已经成功地完成了这个工作，首先，我将我在构建容器中使用的对接器的版本进行了回归，以尝试匹配Btrfs存储驱动程序库版本(到101)。所以..gitlab ci.yml与：
docker-build:
image: docker:17.03.1-dind
stage: build
services:
- name: docker:17.03.1-dind并且容器按其应有的方式构建额外的层，但是构建容器中的docker info显示正在使用的存储驱动程序默认为vfs。
Server Version: 17.03.1-ce
Storage Driver: vfs然后我将版本切换回20.10.3，并强制使用vfs存储驱动程序：
variables:
[SNIP]
DOCKER_DRIVER: vfs
docker-build:
image: docker:20.10.3-dind
stage: build
services:
- name: docker:20.10.3-dind现在起作用了。afaik vfs驱动程序并不理想，但是这个/我的用例，性能不是一个问题。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/74027876
复制相关文章
相似问题
腾讯云开发者
