文档章节

基于Jenkins和Koji的代码管理及构建系统

宅蓝三木
 宅蓝三木
发布于 2014/10/16 15:10
字数 921
阅读 274
收藏 5

<h1>Summary</h1>

Currently, we KVM team are maintaining the code repository of the virtualization software(such as QEMU, Libvirt, Kimchi, Linux, etc.) for PowerKVM development team and delivering corresponding RPM packages. We collect patches from mailing list and build update RPM packages every workday. The work load is extremely heavy if we manually download the patches from mailing list, apply them to the repository, and build RPM packages covering all the supported platforms.

This environment, based on Jenkins and Koji, can automatically manage the code(apply the patches from mailing lists to the appropriate repository and branch) and build the required RPM packages covering all supported platforms nightly or triggered manually. By using this automation environment, developers can easily get the updated RPM packages just by sending the patches to the given e-mail address or mailing list. Furthermore, the time cost for daily routine work is dramatically reduced and the quality of the packages will be improved by eliminating human errors. This environment also can be used to other projects by making few changes to configure files.

<h1>Challenges</h1>

In PowerKVM development, we have to ship all the significant bug fixes frequently, usually a couple days. All the fix patches of QEMU, Libvirt, Linux, Kimchi will be sent to a certain mailing list. After the code cut-off day, we download all the patches from the mailing list and manually apply them to different git repositories accordingly. Then copy all the git repositories to different platform(RHEL,Fedora,etc) and do build. Last step, copy all the output RPM packages to different RPM repositories accordingly. Sometimes, we can not deliver the packages on time because the workload is extremely heavy. Furthermore, errors occur frequently such as some patches may get lost during the process due to lack of time. In order to solve these problems, we decided to make use of Jenkins and Koji to establish the CI environment.

During the construction of this environment, we have to resolve the following problems:

  • Define the rules of sending patches for developers in order to make the CI know where the patches should go.
  • Define the regular expression to distinguish patches from other e-mails.
  • How to handle exceptions.

<h1>Experience</h1>

<1>. Tools and Framework Overview

In order to implement this automation environment, several tools are needed:

  • Jenkins: Jenkins is a software that monitors executions of repeated jobs. This automation environment use Jenkins as the user interface and the manager of all the underlying jobs.
  • Koji: Koji can build RPMs packages for multiple platforms. This automation environment make use of Koji to build RPMs for all supported platform.
  • Python scripts: implement the function that collect patches from mailing list, organize and archive these patches.
  • Shell scripts: Apply the patches to the appropriate git repositories, process build and release the output RPM packages via web server.
  • Apache HTTP server: Release the RPMs via web page. Below is the chart about the framework overview of this automation environment and the relationship among these tools.

Framework

<2>. User Interface

The automation environment runs nightly. However, if the developer want to get the RPM packages immediately, he or she can simply click the start button of Jenkins. See below chart.

User Interface

<3>. Execution flow

After the developers click the first run button, jenkins will call python scripts to collect emails matching certain regular expression(We advise developers send patches to mailing list in fixed forms, such as [branch_name PATCH repo_name]xxx) and archive all the patches by date. Then, Jenkins will call shell scripts to apply these patches to the appropriate git repositories. After that, Koji will be noticed to build the RPM packages of all the supported platforms. Finally, Jenkins will call the shell scripts to copy all the RPM packages to certain directory of Apache web server.

<4>. Handle exception

The automation environment will send report containing how many patches received and where the RPM packages are published to the mailing list if all the jobs are successfully executed. If a certain job failed, it also sends report to the mailing list or email boxes configured in the configuration file. Different job has different error report message. For example, when the job who collects patches from mailing list fails, it will send out the failed patches' name, but when the job who builds RPM packages fails, it will send out the build log.

<h1>Benefit</h1>

For the repositories maintainers, this automation environment can save much time costs for their daily routing work. What they need to do is just check reports via email clients if this automation environment start to work. The time costs dramatically reduce from several hours to a few minutes.

For the developers, they can get the RPM packages by themselves whenever they want to build one, instead of asking the maintainers spend much time to build for them. What they need to do is simply click a button.

Furthermore, the quality of the RPM packages will be improved because this automation environment can eliminate human errors effectively.

<h1>Next Steps and Recommendations</h1>

Currently, this automation environment is just used for code management and RPM packages build. Since it can save much time cost for developers and repository maintainers, we are planing to involve test cases. We can make use of automation test technology and integrate the test cases into this automation environment. After that, test engineers will be set free from burdensome test work.

© 著作权归作者所有

宅蓝三木
粉丝 33
博文 52
码字总数 58473
作品 0
海淀
后端工程师
私信 提问
RPM 构建和跟踪系统--Koji

Koji 是个基于 RPM 的构建系统,主要任务是提供一个灵活,安全和可复用的方法来构建软件。它是个极简,便携的命令行客户端,用户可以创建本地 buildroots 等等。Koji 使用 Mock 来创建 chro...

叶秀兰
2014/03/23
495
0
基于Jenkins 快速搭建持续集成环境

持续集成概述 什么是持续集成 随着软件开发复杂度的不断提高,团队开发成员间如何更好地协同工作以确保软件开发的质量已经慢慢成为开发过程中不可回避的问题。尤其是近些年来,敏捷(Agile)...

openthings
2015/12/09
440
0
基于 Jenkins 快速搭建持续集成环境

简介: 持续集成是一种软件开发实践,对于提高软件开发效率并保障软件开发质量提供了理论基础。Jenkins 是一个开源软件项目,旨在提供一个开放易用的软件平台,使持续集成变成可能。本文正是...

IBMdW
2011/11/24
21.5K
8
Jenkins入门系列之——01第一章 Jenkins是什么?

第一章 Jenkins是什么? Jenkins 是一个可扩展的持续集成引擎。 主要用于: l 持续、自动地构建/测试软件项目。 l 监控一些定时执行的任务。 Jenkins拥有的特性包括: l 易于安装-只要把jen...

雨飞飞
2016/02/26
86
0
有了Jenkins,为啥还需要一个独立的部署系统?

作者介绍 徐桂林 当前在FIT2CLOUD负责公司的技术布道和生态合作。在此之前先后供职于意法半导体、Autodesk和阿里云。徐桂林热衷于云计算(尤其是公有云IaaS平台),有过多年AWS的生产环境工作...

luoahong
2016/01/26
0
0

没有更多内容

加载失败,请刷新页面

加载更多

Handler消息传递机制分析

Handler的用途和用法 写过Android程序的人大概都会遇到ANR(Application Not Responding)。如果程序在一段时间内没有响应,系统就会弹出一个对话框,让用户选择继续等待还是强制关闭应用。为...

tommwq
今天
6
0
JS前端MD5加密

Bootstrap官网获得md5 js地址:https://www.bootcdn.cn/blueimp-md5/ <!--MD5加密--><script src="https://cdn.bootcss.com/blueimp-md5/2.10.0/js/md5.js"></script> 使用方法: md5(pwd)......

被毒打的程序猿_先瑞
今天
8
0
BigDecimal 去后面无用的0的方法

BigDecimal a=new BigDecimal("0.1000"); System.out.println(a.stripTrailingZeros().toPlainString());...

xiaodong16
今天
7
0
JAVA--高级基础开发

[集合版双色球] 十二、双色球规则:双色球每注投注号码由6个红色球号码和1个蓝色球号码组成。红色球号码从1—33中选择;蓝色球号码从1—16中选择;请随机生成一注双色球号码。(要求同色号码...

李文杰-yaya
昨天
27
0
聊聊rocketmq broker的CONSUMER_SEND_MSG_BACK

序 本文主要研究一下rocketmq broker的CONSUMER_SEND_MSG_BACK CONSUMER_SEND_MSG_BACK rocketmq/common/src/main/java/org/apache/rocketmq/common/protocol/RequestCode.java public class......

go4it
昨天
7
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部