全能詹：Jenkins收集测试误报用例

DevOps云学堂

发布于 2019-11-28 21:11:10

1K0

发布于 2019-11-28 21:11:10

前言

什么是误报？这里定义为不是用例本身错误导致的错误（网络、环境等原因造成的失败）。

对于自动化接口测试之前也没有太多的研究，经历了一些实践。先通过Jenkins完成自动化测试框架的搭建和集成（ant+jmeter），然后不断优化将jmeter 纳入到了容器中。其大概过程是通过ant批量的执行测试脚本，最后在jenkins中生成测试报告

关于测试报告中的内容包含以下信息，成功和失败用例的信息。我们先根据接口的返回过滤出一些误报的失败用例（类似于网络错误、环境问题等）。然后通过Jenkins的交互的展示给用户，供用户再次选择其他接口是否也属于误报。确认后将误报的用例计入成功用例数，重新计算成功率和失败率以及误报率。（这肯定不是最佳的做法，最佳的做法应该是有一个测试的平台或系统，当前做法也算是可以完成误报率的统计）

准备实践

需要准备内容

通过jmeter生成测试报告。
通过Jenkins解析测试报告。

生成测试报告

打开jmeter然后创建一个测试用例（这里故意将地址写错）

资源地址：https://github.com/zeyangli/jmetertest.git

运行测试

ZeyangdeMacBook-Pro:jmetertest zeyang$ ls
README.md        jmeter.results.shanhe.me.xsl
build.xml        result
jmeter-results-detail-report_21.xsl  scripts
jmeter.log
ZeyangdeMacBook-Pro:jmetertest zeyang$ ant -buildfile ./build.xml
Buildfile: /Users/zeyang/Desktop/jmetertest/build.xml

run:

test:
   [jmeter] Executing test plan: /Users/zeyang/Desktop/jmetertest/scripts/blog.jmx ==> /Users/zeyang/Desktop/jmetertest/result/jtlfile/TestReport_201911241112.jtl
   [jmeter] 2019-11-24 11:12:31,449 main ERROR FileManager (jmeter.log) java.io.FileNotFoundException: jmeter.log (Permission denied) java.io.FileNotFoundException: jmeter.log (Permission denied)
   [jmeter] Creating summariser <summary>
   [jmeter] Created the tree successfully using /Users/zeyang/Desktop/jmetertest/scripts/blog.jmx
   [jmeter] Starting the test @ Sun Nov 24 11:12:31 CST 2019 (1574565151991)
   [jmeter] Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
   [jmeter] summary +      1 in 00:00:00 =    4.3/s Avg:    34 Min:    34 Max:    34 Err:     1 (100.00%) Active: 3 Started: 3 Finished: 0
   [jmeter] summary +      3 in 00:00:01 =    3.9/s Avg:   338 Min:     0 Max:   807 Err:     1 (33.33%) Active: 0 Started: 3 Finished: 3
   [jmeter] summary =      4 in 00:00:01 =    4.0/s Avg:   262 Min:     0 Max:   807 Err:     2 (50.00%)
   [jmeter] Tidying up ...    @ Sun Nov 24 11:12:33 CST 2019 (1574565153497)
   [jmeter] ... end of run

report:
     [xslt] Processing /Users/zeyang/Desktop/jmetertest/result/jtlfile/TestReport_201911241112.jtl to /Users/zeyang/Desktop/jmetertest/result/htmlfile/201911241112_DetailReport.html
     [xslt] Loading stylesheet /usr/local/apache-jmeter-5.1.1/extras/jmeter.results.shanhe.me.xsl
     [xslt] Processing /Users/zeyang/Desktop/jmetertest/result/jtlfile/TestReport_201911241112.jtl to /Users/zeyang/Desktop/jmetertest/result/htmlfile/201911241112_SummaryReport.html
     [xslt] Loading stylesheet /usr/local/apache-jmeter-5.1.1/extras/jmeter-results-detail-report_21.xsl

BUILD SUCCESSFUL
Total time: 4 seconds

查看测试报告效果

找到生成的测试报告，上传到github。（非必须上传github）

后面就需要Jenkins去下载测试报告并解析了

解析测试报告

准备通过groovy 正则表达式完成HTML报告的分析，发现共同特征。

编写Jenkinsfile

#!groovy
def falsePositive = []

node {
    stage("GetCode"){
        //checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '24982560-17fc-4589-819b-bc5bea89da77', url: 'https://github.com/zeyangli/jmetertest.git']]])
        sh " curl https://raw.githubusercontent.com/zeyangli/jmetertest/master/SummaryReport.html -o SummaryReport.html"
        sh "cat SummaryReport.html"
        publishHTML([allowMissing: false, alwaysLinkToLastBuild: false, keepAll: false, reportDir: '', reportFiles: 'SummaryReport.html', reportName: '接口测试报告', reportTitles: ''])
        
    }

    stage("fenxi"){
        def fileContext = readFile "./SummaryReport.html"
        println(fileContext)
        def pattern = "<h3>.*?<a"
        def matcher = fileContext =~ pattern

        if (matcher.find()){
            println(matcher)

            matcher.each {
                println(it - "<h3>" - "<a")
                falseCase = it - "<h3>" - "<a"
                falsePositive.add(falseCase)
            }
        }
        println(falsePositive)
        
    }

    stage("UserAccept"){

        def result = input  message: '请选择属于误报的用例', 
                            ok: '提交',
                            parameters: [extendedChoice( description: '误报的失败用例', 
                                                         descriptionPropertyValue: '', 
                                                         multiSelectDelimiter: ',', 
                                                         name: 'failePositiveCases', 
                                                         quoteValue: false, 
                                                         saveJSONParameterToFile: false, 
                                                         type: 'PT_CHECKBOX', 
                                                         value: "${falsePositive.toString() - "[" - ']'}", 
                                                         visibleItemCount: 99)]

        println(result)

    }  
}

运行效果