服务端持续集成实战

2020/03/23 15:25
阅读数 709

前言

   基于Jenkins的服务端持续集成已在搜狗商业产品系统实现,实施流程如下图,今天介绍如何在服务端实施持续集成。

Jenkins工程配置

1.新建Jenkins Pipline工程

New Item -> Pipline

2.增加以下Params

ID Tpye Name Description
1 UnitTest Boolean Parameter 是否执行单元测试
2 Branch Git Parameter Git分支
3 RunSonar Boolean Parameter 是否静态代码扫描
4 ApiTest Boolean Parameter 是否需要接口测试
5 ServerIps String Parameter 部署环境IP
6 RemoteU String Parameter 部署用户名

3.填入Git地址和JenkinsFile名称

配置文件创建或修改

1.build.xml

<?xml version="1.0" encoding="UTF-8"?><project xmlns:jacoco="antlib:org.jacoco.ant" default="jacoco" name="Jacoco">  <!--Jacoco 的安装路径,需要放在ant程序所在的目录,否则会报错-->  <property name="jacocoantPath"    value=""/>  <!--最终生成 .exec 文件的路径,Jacoco 就是根据这个文件生成最终的报告的-->  <property name="integrationJacocoexecPath" value="./jacoco.exec"/>  <!--合并后exec文件-->  <property name="allJacocoexecPath" value=""/>  <!--待合并的exec文件-->  <property name="baseDir" value=""/>  <!--生成覆盖率报告的路径-->  <property name="reportfolderPath" value=""/>  <!--源代码路径-->  <property name="checkOrderSrcPath" value="src/main"/>  <!--.class 文件路径-->  <property name="checkOrderClasspath" value="build/classes"/>
<!--让 ant 知道去哪儿找 Jacoco--> <target name="dump"> <jacoco:dump address="" append="false" destfile="${integrationJacocoexecPath}" port="8044" reset="true"/> </target>
<!--dump 任务: 根据配置的 Ip 地址,和端口号, 访问目标 Tomcat 服务,并生成 .exec 文件。--> <taskdef resource="org/jacoco/ant/antlib.xml" uri="antlib:org.jacoco.ant"> <classpath path="${jacocoantPath}"/>  </taskdef></project>

2.sonar-project.properties

sonar.projectKey=rtbmanager:1.0# this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1.sonar.projectName=rtbmanager-1.0sonar.projectVersion=1.0
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.# This property is optional if sonar.modules is set.sonar.sources=srcsonar.exclusions=**/test/**,**/target/**sonar.java.binaries=build/classes/sonar.java.source=1.8sonar.java.target=1.8# Encoding of the source code. Default is default system encodingsonar.sourceEncoding=UTF-8# Set jacoco Configuration# coveragesonar.core.codeCoveragePlugin=jacoco# exec binary filessonar.jacoco.reportPaths=build/jacoco-ut.exec# Path to the JaCoCo report file containing coverage data by integration tests(集成测试). # The path may be absolute or relative to the project base directorysonar.jacoco.itReportPath=cifiles/jacoco.exec# additionalsonar.dynamicAnalysis=reuseReports#sonar.jacoco.reportMissing.force.zero=falsesonar.coverage.exclusions=**/rtb/manager/config/**

3.Java配置文件dev.conf

JAVA_OPTS="-javaagent:/search/odin/jacocoagent.jar=includes=com.sogou.*,output=tcpserver,port=8044,address=127.0.0.1,append=true -Xverify:none"


4.build.gradle/pom.xml

build.gradle

//Jacocoapply plugin: "jacoco" jacoco {    toolVersion = "0.8.3"    reportsDir = file("$buildDir/customJacocoReportDir")}jacocoTestReport {    reports {        xml.enabled false        csv.enabled false        html.destination file("${buildDir}/jacocoHtml")    }}test {    jacoco {        destinationFile = file("$buildDir/jacoco-ut.exec")        classDumpDir = file("$buildDir/classpathdumps")    }}

Jenkins Pipline文件修改

1.Build Stage修改

对于gradle工程来说,单元测试的执行在编译过程就会执行。

./gradlew build -Pprofile=${profile}

该命令即可在编译过程执行单元测试,单元测试通过编译成功,反之失败。

2.UnitTest Stage修改

该stage用于单元测试代码覆盖率统计。
修改classPattern参数,改为对应工程需要统计覆盖率类的目录。

3.SonarQube Scan Stage

该stage将编译后的程序提交至SonarQube,并根据SonarQube返回的结果判定该本次pipline的执行是否成功
SonarQube Scanner的使用方式有两种,

Jenkins插件模式

已安装SonarQube Scanner插件

def scannerHome = tool 'sonarqube_scanner';       sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties"

Jenkins机器手动安装SonarQube Scanner程序(需要在Jenkins机器安装SonarQube Scanner)

sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner  -D project.settings=cifiles/sonar-project.properties'

4.ApiTest Stage

stage 用于执行接口自动化用例,同时统计其覆盖率,并与单元测试覆盖率合并,最终的覆盖率结果在SonarQube上展现

1.修改build job: '{project}-apitest'

2.修改Ant执行方式

Ant的使用方式有两种,插件模式和手动安装模式,更推荐插件模式,以下是两种模式的代码信息

插件模式

withAnt(installation: 'ant'){         sh 'ant dump -buildfile cifiles/build.xml'     }

直接安装

sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile cifiles/build-dispatcher.xml'

Jenkins Pipline样例

#!groovynode {    def tomcat_health_suf = ':8080/actuator/health'    def nginx_health_suf = ':80/actuator/health'    def max_secs = 30    stage('git clone') {        checkout scm    }     stage('gradle build') {        sh './gradlew build -Pprofile=${profile}'    }
stage('UnitTest'){ if (unittest == 'true'){ echo "deploy jacoco......" for (ip in serverIps.split(',')) { if (deployJacoco(ip, remote_user) == false) { echo '[FAILURE] Failed to deploy jacoco' currentBuild.result = 'FAILURE' return } }  echo "starting unitTest jacocoCoverage......" /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/ jacoco( buildOverBuild: false, changeBuildStatus: false, execPattern: 'build/jacoco-ut.exec', classPattern: 'build/classese', sourcePattern: 'src/main/java', exclusionPattern: 'src/test*', minimumMethodCoverage: '10', maximumMethodCoverage: '90', minimumClassCoverage: '40', maximumClassCoverage: '90', minimumLineCoverage: '5', maximumLineCoverage: '90' ) } }  stage('SonarQube Scan') { if (runSonar == 'true') { withSonarQubeEnv('sonarqube') { //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同 echo "starting codeAnalyze with SonarQube......" echo "WORKSPACE: $WORKSPACE" //执行Sonar Scanner def scannerHome = tool 'sonarqube_scanner'; sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties" // sh '/search/odin/sonar/sonar-scanner-cli/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties' } script { timeout(1) { //这里设置超时时间1分钟,如果Sonar Webhook失败,不会出现一直卡在检查状态 //利用Sonar webhook功能通知pipeline代码检测结果,未通过质量阈,pipeline将会fail def qg = waitForQualityGate('sonarqube') //注意:这里waitForQualityGate()中的参数也要与之前SonarQube servers中Name的配置相同 if (qg.status != 'OK') { error "未通过Sonarqube的代码质量阈检查,请及时修改!failure: ${qg.status}" } } } } } if (confirming == 'true') { stage('staging') { if (deploy(stagingIp, remote_user, "staging", tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) { echo '[FAILURE] Failed to build' currentBuild.result = 'FAILURE' return } } } stage('deploy') { if (confirming == 'true') { input 'make sure to publish?' } for (ip in serverIps.split(',')) { if (deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) == false) { echo '[FAILURE] Failed to build' currentBuild.result = 'FAILURE' return } } } stage('ApiTest'){ if (apiTest == 'true') { build job: 'rtbmanager-apitest' echo 'starting ant dump......' withAnt(installation: 'ant'){ sh 'ant dump -buildfile cifiles/build.xml' } // sh '/usr/local/software/apache-ant-1.9.14/bin/ant dump -buildfile /usr/local/software/apache-ant-1.9.14/build-rtbmanager.xml' echo 'starting apiTest jacocoCoverage......' /*jacoco changeBuildStatus: true, maximumLineCoverage:"80"*/ jacoco( buildOverBuild: false, changeBuildStatus: false, execPattern: 'cifiles/jacoco.exec', classPattern: 'build/classes', sourcePattern: 'src/main/java', exclusionPattern: 'src/test*', minimumMethodCoverage: '70', maximumMethodCoverage: '90', minimumClassCoverage: '80', maximumClassCoverage: '90', minimumLineCoverage: '60', maximumLineCoverage: '90' ) withSonarQubeEnv('sonarqube') { //注意这里withSonarQubeEnv()中的参数要与之前SonarQube servers中Name的配置相同 echo "starting codeAnalyze with SonarQube......" //统计接口测试覆盖率,并同步至SonarQube def scannerHome = tool 'sonarqube_scanner'; sh "${scannerHome}/bin/sonar-scanner -D project.settings=cifiles/sonar-project.properties" } } }} def deploy(ip, remote_user, profile, tomcat_health_suf, nginx_health_suf, max_secs, nginx) { if (nginx == 'true') { sh "ssh ${remote_user}@${ip} \"sudo systemctl stop openresty.service\"" } sh "scp build/libs/app.jar ${remote_user}@${ip}:/search/odin/app/app.jar" sh "scp envfiles/${profile}.conf ${remote_user}@${ip}:/search/odin/app/app.conf" sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\"" if (check(ip + tomcat_health_suf, max_secs, 8) == false) { return false } if (nginx == 'true') { sh "ssh ${remote_user}@${ip} \"sudo systemctl start openresty.service\"" return check(ip + nginx_health_suf, max_secs, 1) } return true} def deployJacoco(ip, remote_user) { sh "scp -r cifiles/jacocoagent.jar ${remote_user}@${ip}:/search/odin/" sh "ssh ${remote_user}@${ip} \"sudo systemctl restart myapp.service\"" return true} def check(url, max, initial_sleep_secs) { if (!url.startsWith("http")) { url = 'http://' + url } def cmd = $/eval "curl -s ${url} | sed 's/ //g' | grep '\"status\":\"UP\"' | wc -l"/$ echo "${cmd}" def rc = "0"; sleep(initial_sleep_secs); // seconds try { rc = sh( script: "${cmd}", returnStdout: true ).trim(); echo rc } catch (Exception e) { } def i = 0; while (rc.equals("0") && i < max) { sleep(1); // seconds try { rc = sh( script: "${cmd}", returnStdout: true ).trim(); echo rc } catch (Exception e) { } i++; } echo rc return rc.equals("0") ? false : true;}

服务端持续集成效果展示

Jenkins持续集成构建结果:

Jenkins持续集成邮件通知:





搜狗测试微信号:Qa_xiaoming

搜狗测试QQ粉丝群:459645679

本文分享自微信公众号 - 搜狗测试(SogouQA)。
如有侵权,请联系 support@oschina.cn 删除。
本文参与“OSC源创计划”,欢迎正在阅读的你也加入,一起分享。

展开阅读全文
加载中
点击引领话题📣 发布并加入讨论🔥
0 评论
0 收藏
0
分享
返回顶部
顶部