gitlab实现CICD的实战过程

用户10002156

发布于 2024-01-03 15:01:53

2600

发布于 2024-01-03 15:01:53

文章被收录于专栏：生活处处有BUG生活处处有BUG

背景

目前的现状,开发者在提交代码后还需要去构建镜像,上传镜像到镜像仓库,频繁的修改就需要频繁的构建。

• 用户每次写完代码要本地构建,需要本地有docker运行的环境,环境问题在日常开发中经常出现.
• 本地构建、推送到仓库后本地就残留了镜像文件,因为仓库会保存镜像,本地镜像就会浪费本地的存储空间.
• 代码需要通过代码检测平台的检测,比如社区的一些优秀解决方案 : SonarQube等
• push完代码后(或者push之前)代码需要有一些自动化测试

安装

前置条件

需要安装docker和docker-compose(docker安装和docker-compose社区有大量的文章,本文不做说明)

搜索并下载gitlab和gitlab-runner的镜像

docker pull gitlab/gitlab-ee:14.5.2-ee.0
docker pull gitlab/gitlab-runner

清除之前的测试数据（要小心你的环境下这个目录下没有生产或重要数据，是可以自由删除的）

sudo rm -rf ~/software/docker/gitlab/
mkdir ~/software/docker/gitlab/ && cd ~/software/docker/gitlab/

新建demo文件夹，并在demo文件夹下新建docker-compose.yml文件

echo -e "version: '3.3'

services:
  gitlab:
    container_name: gitlab
    image: gitlab/gitlab-ee:14.5.2-ee.0
    restart: always
    hostname: 10.147.20.17
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://10.147.20.17'
        # Add any other gitlab.rb configuration here, each on its own line
    ports:
      - 80:80
      - 7443:443
    volumes:
      - ~/software/docker/gitlab/config:/etc/gitlab
      - ~/software/docker/gitlab/logs:/var/log/gitlab
      - ~/software/docker/gitlab/data:/var/opt/gitlab" > docker-compose.yml

docker-compose up -d

在另一台电脑，也可以是同一台电脑

sudo rm -rf ~/software/docker/gitlab-runner/
mkdir ~/software/docker/gitlab-runner/ && cd ~/software/docker/gitlab-runner/

echo "version: '3.3'
services:
  gitlab-runner:
    container_name: gitlab-runner
    image: gitlab/gitlab-runner
    restart: unless-stopped
    privileged: true
    volumes:
      - ~/software/docker/gitlab-runner/data:/etc/gitlab-runner
      - /home/xj/tmp/1023/test:/home/xj/tmp/1023/test
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/bin/dockerr
    networks:
      - mynetwork
networks:
  mynetwork:
    external: true" > docker-compose.yml

• /var/run/docker.sock:/var/run/docker.sock 宿主机机的docker.sock映射到镜像里面
• /usr/bin/docker:/bin/docker 宿主机的docker可执行映射到镜像里面后面build的时候会用到

在当前目录运行命令，这个container启动需要些时间，等待一会就好

docker-compose up -d

gitlab-ce初装以后，密码放在了一个临时文件中 /etc/gitlab/initial_root_password 这个文件将在首次执行reconfigure后24小时自动删除

cicd0安装成功

查看gitlab-ce的root账号的默认密码（copy保存好，待会要登录用）

docker exec -it gitlab cat /etc/gitlab/initial_root_password

image-20231020191823670

登录你的IP，如：http://10.147.20.17 （这里换成你自己的ip)

image-20231021122709663

账号是root，密码是刚查看的密码

修改成中文

image-20231022214641045

保存之后，刷新一下即可显示中文界面

点击上边的加号，新建一个代码仓库

image-20231021123119798

开始你的第一个runner

注册runner

进入项目设置->CI/CD->Runner

image-20231022214847737

(base) xj@xjpc:~/software/docker/gitlab-runner$ docker exec -it gitlab-runner gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=28 revision=853330f9 version=16.5.0
Running in system-mode.                            
                                                   
Enter the GitLab instance URL (for example, https://gitlab.com/):
http://10.147.20.17/
Enter the registration token:
xKw1V6r38xd72xrdCqFm
Enter a description for the runner:
[1ed3682970c5]: 
Enter tags for the runner (comma-separated):
xjpc
Enter optional maintenance note for the runner:

WARNING: Support for registration tokens and runner parameters in the 'register' command has been deprecated in GitLab Runner 15.6 and will be replaced with support for authentication tokens. For more information, see https://docs.gitlab.com/ee/ci/runners/new_creation_workflow 
Registering runner... succeeded                     runner=xKw1V6r3
             
Enter an executor: ssh, virtualbox, docker+machine, instance, custom, docker-windows, parallels, shell, docker-autoscaler, kubernetes, docker:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
 
Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"

查看配置

docker exec -it gitlab-runner cat /etc/gitlab-runner/config.toml

把gitlab-runner加入到sudo

docker exec -it gitlab-runner usermod -aG sudo gitlab-runner

检测

这个时候我们再回到gitlab页面上,就能看到刚才我们最新注册的runner(我之前注册过所以这边显示两个)

image-20231022215231911

至此我们的注册步骤已经结束了当我们再向仓库push代码的时候就会根据我们的仓库根目录的gitlba-ci.yml文件运行我们的CI,下面我将贴一下一个最简单的gitlba-ci.yml文件

CICD配置

进入仓库的CI/CD,点击编辑器，点击配置流水线，这里有个deom配置模板。

注意

配置中一定要选好执行的runner，就是要写好runner对应的tags，如上边截图的xjpc，简单的配置demo如下：

# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Getting-Started.gitlab-ci.yml

# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/index.html#stages

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  tags:
    - xjpc
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - echo "Code coverage is 90%"

查看执行情况

image-20231022215654312

点进去，可以查看执行的过程

image-20231022215735471

至此，大概的配置就完成了。

其他

demo:git clone

docker exec -it -u gitlab-runner gitlab-runner ssh-keygen
docker exec -it -u gitlab-runner gitlab-runner ssh-copy-id -i /home/gitlab-runner/.ssh/id_rsa.pub xj@172.16.101.222

gitlba-ci.yml配置（一般是在runner上编译和测试，而不是ssh到生产服务器去编译）

# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Getting-Started.gitlab-ci.yml

# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/index.html#stages

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  tags:
    - xjpc
  script:
    - echo $USER
    - ssh xj@172.16.101.222 "cd /home/xj/tmp/1023/test;git pull http://root:Xiaojin123.@10.147.20.17/root/test.git"

demo:rsync

因该文档选择的执行程序为 shell ，故采用 rsync 命令进行目录同步，如有多台机器新增配置即可。如对 gitlab-runner 进行注册时选择执行程序为 docker 等，该配置并不适用。

before_script:
    - date
stages:
    - build
    - test
    - deploy

deploy_in_web1:
    stage: deploy
    script:
        - git checkout master
        - git pull
        - rsync -rvz --no-owner --no-group --no-perms --progress --exclude=".*" --exclude="/vendor" --delete $CI_PROJECT_DIR/ /data/项目路径
    only:
        - master
    tags:
        - "web1"
        

#配置多台机器，新增配置即可
deploy_in_web2:
    stage: deploy
    script:
        - git checkout master
        - git pull
        - rsync -rvz --no-owner --no-group --no-perms --progress --exclude=".*" --exclude="/vendor" --delete $CI_PROJECT_DIR/ /data/项目路径
    only:
        - master
    tags:
        - "web2"

流水线配置

before_script:
  - echo '====== 准备构建中 ========='

# 单个job全部执行完之后执行
after_script:
  - echo "====== 构建结束 ========="

推荐CI/CD 配置

项目、系统、环境等不尽相同，推荐部署Laravel项目按照的dev，test，production三个环境构建项目，仓库分支保持develop，testing，master三个分支对应前面三个环境构建代码，每个环境的应用参数也不同，可以采用.env.dev，.env.test，.env.production保存参数。

variables:
  RELEASES_STORAGE_DIR: '/var/www/$CI_COMMIT_REF_NAME/$CI_PROJECT_PATH/storage'
  CREATE_RELEASES_STORAGE_DIR: '[ -d $RELEASES_STORAGE_DIR ] || sudo mkdir -p $RELEASES_STORAGE_DIR'
  RELEASES_DIR: '/var/www/$CI_COMMIT_REF_NAME/$CI_PROJECT_PATH/releases'
  CREATE_RELEASE_DIR: '[ -d $RELEASES_DIR ] || sudo mkdir -p $RELEASES_DIR'
  NEW_RELEASES_DIR: '$RELEASES_DIR/$CI_COMMIT_SHORT_SHA'
  CREATE_NEW_RELEASES_DIR: '[ -d $NEW_RELEASES_DIR ] || sudo mkdir -p $NEW_RELEASES_DIR'
  BEFORE_CHMOD: 'sudo chown -R deployer:deployer $NEW_RELEASES_DIR'
  BEFORE_CHMOD_VENDOR: 'sudo chown -R deployer:deployer $NEW_RELEASES_DIR/vendor'
  AFTER_CHMOD: 'sudo chown -R apache:apache /var/www/$CI_COMMIT_REF_NAME && sudo chown -R apache:apache $RELEASES_STORAGE_DIR && sudo chmod -R 777 $RELEASES_STORAGE_DIR'
  CD_NEW_RELEASES_DIR: 'cd $NEW_RELEASES_DIR'
  CD_RELEASES_DIR: 'cd $RELEASES_DIR'
  #Linux删除除了某个文件之外的所有文件/目录
  CLEAN_RELEASES_DIR: 'ls |grep -v $CI_COMMIT_SHORT_SHA |xargs sudo rm -rf'
  RM_RELEASE_STORAGE_DIR: 'sudo rm -rf $NEW_RELEASES_DIR/storage'
  LN_RELEASE_STORAGE_DIR: 'sudo ln -nfs $RELEASES_STORAGE_DIR $NEW_RELEASES_DIR/storage'
  LN_RELEASE_DIR: 'sudo ln -nfs $NEW_RELEASES_DIR /var/www/$CI_COMMIT_REF_NAME/$CI_PROJECT_PATH/current'
  MV_REPO: 'sudo mv -fv /home/deployer/$CI_PROJECT_DIR/* $NEW_RELEASES_DIR'
  CP_DEV_ENV: 'cp /home/deployer/$CI_PROJECT_DIR/.env.dev $NEW_RELEASES_DIR/.env'
  CREATE_FRAMEWORK_CACHE: '[ -d $RELEASES_STORAGE_DIR/framework/cache ] || sudo mkdir -p $RELEASES_STORAGE_DIR/framework/cache'
  CREATE_FRAMEWORK_SESSIONS: '[ -d $RELEASES_STORAGE_DIR/framework/sessions ] || sudo mkdir -p $RELEASES_STORAGE_DIR/framework/sessions'
  CREATE_FRAMEWORK_TESTING: '[ -d $RELEASES_STORAGE_DIR/framework/testing ] || sudo mkdir -p $RELEASES_STORAGE_DIR/framework/testing'
  CREATE_FRAMEWORK_VIEWS: '[ -d $RELEASES_STORAGE_DIR/framework/views ] || sudo mkdir -p $RELEASES_STORAGE_DIR/framework/views'


before_script:
  - echo "Before script"
  - echo $CI_COMMIT_REF_NAME
  - echo $CI_PROJECT_PATH
  - echo $CI_COMMIT_SHORT_SHA
  - echo $CI_REPOSITORY_URL
  - echo $CI_PROJECT_DIR
  - 'eval $CREATE_RELEASES_STORAGE_DIR'  # will execute
  - 'eval $CREATE_RELEASE_DIR'  # will execute
  - 'eval $CREATE_NEW_RELEASES_DIR'  # will execute
  - 'eval $CD_NEW_RELEASES_DIR'


stages:
  - build
  - test
  - deploy-dev

building:
  stage: build
  script:
    - echo "Move repo..."
    - echo $NEW_RELEASES_DIR
    - 'eval $BEFORE_CHMOD'
    - 'eval $MV_REPO'
    - composer install
    - 'eval $BEFORE_CHMOD_VENDOR'

testing:
  stage: test
  script:
    - echo "testing..."
    # - php ./vendor/bin/phpunit

deploying_dev:
  stage: deploy-dev
  script:
    - echo "deploying dev..."
    - 'eval $CP_DEV_ENV'
    - php artisan key:generate
    - 'eval $CREATE_FRAMEWORK_CACHE'
    - 'eval $CREATE_FRAMEWORK_SESSIONS'
    - 'eval $CREATE_FRAMEWORK_TESTING'
    - 'eval $CREATE_FRAMEWORK_VIEWS'
    - php artisan cache:clear
    - php artisan config:clear
    - php artisan storage:link
    - php artisan migrate --force
    - php artisan passport:keys
    - echo "Restarting supervisor"
    - sudo supervisorctl restart all
    - echo "Linking storage directory"
    - 'eval $RM_RELEASE_STORAGE_DIR'
    - 'eval $LN_RELEASE_STORAGE_DIR'
    - echo 'Linking current directory'
    - 'eval $AFTER_CHMOD' 
    - 'eval $LN_RELEASE_DIR'
    - echo 'Removing earlier app'
    - 'eval $CD_RELEASES_DIR'
    - 'eval $CLEAN_RELEASES_DIR'
  only:
    - develop

本文参与腾讯云自媒体分享计划，分享自微信公众号。

原始发表：2023-12-26，如有侵权请联系 cloudcommunity@tencent.com 删除

容器镜像服务