自动化 DevOps 使用 Codeclimate 执行代码质量分析

$ docker pull codeclimate/codeclimate

$ docker run \
>   --interactive --tty --rm \
>   --env CODECLIMATE_CODE="$PWD" \
>   --volume "$PWD":/code \
>   --volume /var/run/docker.sock:/var/run/docker.sock \
>   --volume /tmp/cc:/tmp/cc \
>   codeclimate/codeclimate help
Usage: codeclimate COMMAND ...

Available commands:
    analyze [-f format] [-e engine[:channel]] [path]    Run analysis with the given arguments
    console                                             Open a ruby console for the CLI. Useful for developing against the CLI.
    engines:install                                     Pull the latest images for enabled engines in your configuration
    engines:list                                        List all available engines
    help [command]                                      Display help information.
    prepare [--allow-internal-ips]                      Run the commands in your prepare step.
    validate-config                                     Validate your .codeclimate.yml or .codeclimate.json.
    version                                             Display the CLI version.

# 安装
brew tap codeclimate/formulae
brew install codeclimate
# 更新升级
brew update
brew upgrade codeclimate

curl -L https://github.com/codeclimate/codeclimate/archive/master.tar.gz | tar xvz
cd codeclimate-* && sudo make install

$ codeclimate help
Usage: codeclimate COMMAND ...

Available commands:
    analyze [-f format] [-e engine[:channel]] [path]    Run analysis with the given arguments
    console                                             Open a ruby console for the CLI. Useful for developing against the CLI.
    engines:install                                     Pull the latest images for enabled engines in your configuration
    engines:list                                        List all available engines
    help [command]                                      Display help information.
    prepare [--allow-internal-ips]                      Run the commands in your prepare step.
    validate-config                                     Validate your .codeclimate.yml or .codeclimate.json.
    version                                             Display the CLI version.

$ mkdir test
$ cd test
$ cat test.php
<?php

/**
 * 首页
 */

class MobileController extends AbstractController {

    public $authorize = self::MAYBE_LOGIN;
    public $init_viewer = true;

    public function indexAction() {
        $page = new Page_Mobile_Home();
        $this->render_page($page);

    }
}

# 使用 json 输出方式，使用 phpmd 引擎来执行分析
$ codeclimate analyze -f json -e phpmd
[{"type":"issue","check_name":"Controversial/CamelCasePropertyName","description":"The property $init_viewer is not named in camelCase.","categories":["Style"],"remediation_points":500000,"location":{"path":"test.php","lines":{"begin":8,"end":18}},"content":{"body":"\nSince: PHPMD 0.2\n\nIt is considered best practice to use the camelCase notation to name attributes.\n\n\nExample:\n\n```php\n\n  class ClassName {\n      protected $property_name;\n  }\n\n\n```\n\nSource: http://phpmd.org/rules/controversial.txt"},"fingerprint":"56027498261b4b1ca168bb3fe849c290","engine_name":"phpmd","severity":"minor"}]

$ codeclimate analyze -f html -e phpmd > result.html

$ codeclimate analyze -f html -e phan > result_phan.html

$ docker ps
CONTAINER ID        IMAGE                     COMMAND                  CREATED                  STATUS                  PORTS                                              NAMES
65282e7c8857        codeclimate/codeclimate   "/usr/src/app/bin/co…"   Less than a second ago   Up Less than a second                                                      trusting_mestorf

$ docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
codeclimate/codeclimate                latest              bc2aa56b8520        2 weeks ago         97.7MB
codeclimate/codeclimate-phan           latest              045ea1dac22d        4 months ago        70.9MB
codeclimate/codeclimate-phpmd          latest              9d7a53405aa0        11 months ago       37MB

$ codeclimate engines:list
Available engines:
- apexmetrics: ApexMetrics is a static code analysis tool for Salesforce.com Apex.
- brakeman: A static analysis tool which checks Ruby on Rails applications for security vulnerabilities.
- bundler-audit: Patch-level verification for Bundler.
- checkstyle: Helps programmers write Java that adheres to a coding standard.
- codescan: A static analysis tool for Salesforce.com Apex, Visual Force, Aura and Lightning.
- coffeelint: A style checker for CoffeeScript.
- credo: A static code analysis tool for the Elixir language with a focus on code consistency and teaching.
- csslint: Automated linting of Cascading Stylesheets.
- duplication: Structural duplication detection for Ruby, Python, Java, JavaScript, and PHP.
- eslint: A JavaScript/JSX linting utility.
- fixme: Finds FIXME, TODO, HACK, etc. comments.
- flog: Easy to read reporting of complexity/pain for Ruby code.
- foodcritic: Lint tool for Chef cookbooks.
- git-legal: License compliance and compatibility analysis.
- gnu-complexity: Checks complexity of C code.
- gofmt: Checks the formatting of Go programs.
- golint: A linter for Go.
- govet: Reports suspicious constructs in Go programs.
- grep: Detects specified patterns in files.
- haml-lint: A tool for writing clean and consistent HAML.
- haxe-checkstyle: Checkstyle is a development library to help developers write Haxe code that adheres to a coding standard.
- hlint: A linter for Haskell programs.
- kibit: Static code analyzer for Clojure, ClojureScript, cljx and other Clojure variants.
- markdownlint: Flags style issues within Markdown files.
- nodesecurity: Security tool for Node.js dependencies.
- pep8: Static analysis tool to check Python code against the style conventions outlined in PEP-8.
- phan: Phan is a static analyzer for PHP.
- phpcodesniffer: Detects violations of a defined set of coding standards in PHP.
- phpmd: A PHP static analysis tool.
- pmd: A source code analyzer for Java.
- proselint: A linter for prose.
- radon: Python tool used to compute Cyclomatic Complexity.
- reek: Reek examines Ruby classes, modules, and methods and reports any code smells it finds.
- requiresafe: Security tool for Node.js dependencies.
- rubocop: A Ruby static code analyzer, based on the community Ruby style guide.
- rubymotion: Rubymotion-specific rubocop checks.
- rustfmt: A tool for formatting Rust code according to style guidelines.
- scss-lint: A configurable tool for writing clean and consistent SCSS.
- shellcheck: A static analysis tool for shell scripts.
- sonar-java: Over 400 checks for bugs, vulnerabilities, and code smells in Java code.
- sonar-php: SonarLint for PHP.
- sonar-python: SonarLint for Python.
- structure: Performs structural checks on code.
- stylelint: A mighty, modern CSS linter.
- swiftlint: A tool to enforce Swift style and conventions.
- tailor: Cross-platform static analyzer and linter for Swift.
- tslint: An extensible linter for the TypeScript language
- vint: Fast and Highly Extensible Vim script Language Lint implemented by Python.
- watson: A young Ember Doctor to help you fix your code.

$ cat .codeclimate.yml
version: "1"
plugins:
  phpmd:
    enabled: false
  phan:
    enabled: true
  shellcheck:
    enabled: true

$ cat test.sh
#!/bin/bash

name="This is shell test."
echo "name=${name}"

if [ -z "${version}" ]; then
    echo "version is empty, exit"
    exit 1
fi

$ cat Test.java
package test.hwy;

public class Test {

    public static String getSameEndChart(String[] args){
        String result = "";
        if(args.length > 1){
            int args_length = args.length;
            String first = args[0];
            int first_length = first.length();
            for(int j=1;j <= first_length;j++){
                String a = first.substring(first_length-j, first_length);
                for(int i=1;i < args_length;i++){
                    String b = args[i];
                    int b_length = b.length();
                    if(b_length >= j){
                        String c = b.substring(b_length-j, b_length);
                        if(a.equals(c)){
                            result = a;
                        }else{
                            System.out.println("args 第" + (i+1) + "个字符串尾数字符串"+c+"不匹配首字符串尾字符"+a);
                            result = first.substring(first_length-j+1, first_length);
                            return result;
                        }
                    }else{
                        System.out.println("args 第" + (i+1) + "个字符串长度不够，忽略比较。");
                        result = first.substring(first_length-j+1, first_length);
                        return result;
                    }
                }
            }
        }
        return result;
    }

    public static void main(String[] args) {
        String[] ss = {"123456", "14456", "00123456", "456"};
        System.out.println(getSameEndChart(ss));
    }
}

$ codeclimate analyze -f html > result_yml.html

$ codeclimate engines:install
Pulling docker images.
latest: Pulling from codeclimate/codeclimate-structure
2a72cbf407d6: Already exists
a3ed95caeb02: Pulling fs layer
......
Digest: sha256:b6ee14587a178b1e462202779579affd9186e2be8ca94b141f3c09a3c8862b71
Status: Downloaded newer image for codeclimate/codeclimate-structure:latest
latest: Pulling from codeclimate/codeclimate-duplication
2a72cbf407d6: Already exists
a3ed95caeb02: Pulling fs layer
......
Digest: sha256:a2355a74fb4134b526716a3416b9e841cebbadcfe72f240308686c5ebf424171
Status: Downloaded newer image for codeclimate/codeclimate-duplication:latest
latest: Pulling from codeclimate/codeclimate-phpmd
244d757dc677: Already exists
......
Digest: sha256:799d7eb629cfad2ee35a3f0e8a45fda34edf40bd92a43b0f06ac5505ca81ffc6
Status: Image is up to date for codeclimate/codeclimate-phpmd:latest
latest: Pulling from codeclimate/codeclimate-phan
1160f4abea84: Already exists
......
Status: Image is up to date for codeclimate/codeclimate-phan:latest
latest: Pulling from codeclimate/codeclimate-shellcheck
c19324d1d971: Already exists

$ docker images
REPOSITORY                             TAG                 IMAGE ID            CREATED             SIZE
codeclimate/codeclimate-duplication    latest              a864651fe3d2        7 days ago          2.36GB
codeclimate/codeclimate-structure      latest              d12a28ef9ccc        8 days ago          2.35GB
codeclimate/codeclimate                latest              bc2aa56b8520        2 weeks ago         97.7MB
codeclimate/codeclimate-phan           latest              045ea1dac22d        4 months ago        70.9MB
codeclimate/codeclimate-shellcheck     latest              c46113b2195f        5 months ago        57.8MB
codeclimate/codeclimate-phpmd          latest              9d7a53405aa0        11 months ago       37MB

- duplication: Structural duplication detection for Ruby, Python, Java, JavaScript, and PHP.
- structure: Performs structural checks on code.

$ mkdir others
$ cat others/a.php
<?php
abc;
$array = array(
    "foo" => "bar",
    42    => 24
    "multi" => array(
         "dimensional" => array(
             "array" => "foo"
         )
    )
);

var_dump($array["foo"][]);
var_dump($array[43]);
var_dump($array["multi"]["dimensional"]["array"]);

$ cat a.sh
#!/bin/bash

#./codeclimate.sh > ./result.html

echo $name
codeclimate analyze -f html -e phan

version: "1"
plugins:
  phpmd:
    enabled: false
  structure:
    enabled: false
  phan:
    enabled: true
    exclude_paths:
      - others/a.php
  shellcheck:
    enabled: true
    exclude_paths:
      - a.sh

version: "1"
plugins:
  phpmd:
    enabled: false
  ......
exclude_patterns:
  - others/
  - exclude_dir/**/*
  - exclude_another_dir/**/*

# 例如线上版本为 v1.0.0 本次上线版本 v1.0.1，对本次上线增量文件执行检测，以 qd_api 为例。
$ git diff --name-only v1.0.0 v1.0.1
src/main/java/com/sina/qd/controller/MachineController.java
src/main/java/com/sina/qd/controller/PlatformTransferService.java
src/main/java/com/sina/qd/controller/ProjectInstanceController.java
src/main/java/com/sina/qd/utils/HttpClientUtil.java

# 使用 git archive 命令打包输出两个版本间的改变增量归档
# 大致的命令格式：git archive -o <path>/<filename.filetype> xxx(新提交 commit_id | tag) $(git diff --name-only yyy(旧提交 commit_id | tag) xxx) 
$ git archive -o /data0/latest.tar.gz v1.0.1 $(git diff --name-only v1.0.0 v1.0.1)

$ tar -zxvf /data0/latest.tar.gz -C /data0/latest
$ tree /data0/latest
latest
└── src
    └── main
        └── java
            └── com
                └── sina
                    └── qd
                        ├── controller
                        │   ├── MachineController.java
                        │   ├── PlatformTransferService.java
                        │   └── ProjectInstanceController.java
                        └── utils
                            └── HttpClientUtil.java

8 directories, 4 files

# 执行代码检测
$ cd /data0/latest
$ codeclimate analyze -f html -e sonar-java > result.html

$ cat .gitlab-ci.yml
...
codequality:
  image: docker:stable
  variables:
    DOCKER_DRIVER: overlay2
  allow_failure: true
  services:
    - docker:stable-dind
  script:
    - export SP_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/')
    - docker run
        --env SOURCE_CODE="$PWD"
        --volume "$PWD":/code
        --volume /var/run/docker.sock:/var/run/docker.sock
        "registry.gitlab.com/gitlab-org/security-products/codequality:$SP_VERSION" /code
  artifacts:
    paths: [codeclimate.json]