Contenu connexe Similaire à LCA 2014 project-builder.org presentation (20) Plus de Bruno Cornec (20) LCA 2014 project-builder.org presentation1. Project Builder:
A GPL continuous packaging solution
1
Bruno Cornec
V2.1 6th of January 2014
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
HP EMEA Open Source Profession Lead
2. Introducing Myself
Software engineering and Unices since 1988
●
–
Mostly Configuration Management Systems, Build systems, quality tools, on multiple commercial Unix
systems
–
Discover Open Source & Linux (OSL) & first contributions in 1993
–
Full time on OSL since 1995, first as HP reseller then @HP
Currently:
●
–
Master Technology Architect on OSL for the HP/Intel Solution Center, Grenoble
–
HP OSL Advocate and Converged Infrastructure Ambassador
–
HP EMEA OSL Profession Lead
–
Solutions Linux Conference and AFUL board member. Conferences at WW level in LinuxCon, Linux.conf.au
–
MondoRescue, Project-Builder.org, UUWL and PUSK Project Lead
–
LinuxCOE, mrepo, tellico, rinse, fossology, collectl contributor
–
FOSSBazaar and OSL Governance enthusiast
–
Mandriva, Mageia, Fedora packager
2
●
And also:
–
Amateur singer (Alto / Tenor) and recorder player since 1976 and Choir director since 1987
–
CD collector since 1981 (5000+ and counting) – Concert attendance since 1976
–
Amateur photograph since 1976
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
3. What is project-builder.org’s goal ?
Make upstream projects
life easier with regards to
packaging their software
3
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
4. What is project-builder.org’s itch ?
Make *my*
life easier with regards to
packaging *my* software
4
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
5. Users / Sysadmins want packaged software
●
Ease of use: GUI, CLI
>= 90%
●Distribution compliance
●Smooth Integration with
deployment tools
U
5
< 10%
< 1%
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
6. Users / Sysadmins want packaged software
●
Ease of use: GUI, CLI
>= 90%
●Distribution compliance
●Smooth Integration with
deployment tools
U
< 10%
< 1%
6
D
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
Lag between SW and Pkg
availability
●Test of alpha/beta/VCS SW
●Too many:
● distributions,
● versions,
● package formats,
● tools to generate packages
● tools to manage repositories
●
7. Upstream benefits from Continuous packaging
Packaging should be a project concern as well as coding, testing, installing, ....
especially for smaller projects
Use packaging as your only way of delivery (not a dream)
Minial overhead, slightly longer than providing a tgz, maximum benefit:
Consistancy and reproduceability for devs and users
Distribution & deployment server integration,
Improved deployment without risk of screwing up the system
7
Packaging is also a marketing activity for the upstream project based on a
technical content. It's an easy way to extend your user base, and improve your
community relationship and is a “competitive advantage”.
New mantra: “Package early, package always”
THE SOLUTION IS INDEED CONTINUOUS PACKAGING (whatever the tool)
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
8. Continuous Packaging Architecture
Local Build Server (may host VMs and/or VEs)
Packagers
VM or VE Build
Repository
Server
Build +
metadata
Local build
8
Project
RM Build
Farm
Developers
(may host VMs)
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
Remote build
9. Provisioning the Continuous Packaging Cloud
Local Build Server (may host VMs and/or VEs)
VM or VE Build
Build +
9
metadata
Local build
Project
RM Build Farm (may
host VMs)
Remote build
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
Repository
Server
10. Goals
Project-builder main goal is to help you package continuously being
agnostic:
VCS agnostic: no VCS but guys it's 21st century now, SVN, CVS,
Mercurial, GIT and GIT/SVN, SVK....
OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 150+ distro
tuples made and counting – repositories for yum, urpmi, apt. Solaris
pkg. HP-UX sd in roadmap
10
Build environment agnostic: local, VM (QEMU, KVM...), VE
(rpmbootstrap, rinse, mock, debootstrap...), RM (external build farm)
No project impact: preserves the md5sum of the delivered upstream
sources. Can be completely external to the upstream project.
Avoids duplication of code and metadata
THE SOLUTION IS INDEED CONTINUOUS PACKAGING (with projectbuilder.org !)
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
11. Goodies
Project-builder provides additional goodies:
Easy VMs/VEs/RMs installation/setup. More on that later
Macro system with perl variables to avoid duplication
Skeleton generation to help starting
Manages package delivery up to your repository (ssh based) with
repository management (yum, apt, urpmi)
Manages announces on mailing lists
11
Integration of tests in the process
Manages patches/additional sources when not upstream
Checks validity of packages built (lintian, rpmlint)
Easy creation of new versions for upstream management
Manages website delivery
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
12. VM, VE & RM Management
VE Creation:
debootstrap for .deb based ones.
rpmbootstrap (modeled after debootstrap): create chroot for multiple
RPM based distributions (Mageia, Fedora, Mandriva, CentOS,
OpenSuSE)
VM|RM Creation:
KVM or QEMU iso install possibilities.
12
LinuxCOE/OpenStack as provisioning tool for your VM|RM
VM|VE|RM setup: adds pb tools and build account
VM|VE Snapshot feature – Update of VM|VE|M optional
Building in VM|VE|RM == building locally (build2vm|ve|rm vs build2pkg).
ssh communication for VM/RM
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
13. Metadata Management
Packaging process independent of the project and external
Metadata isolated - No duplication
13
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
14. Configuration file usage
Macro example:
./rpmbootstrap/pbfilter/all.pbf:filter PBSUMMARY = rpmbootstrap is a tool similar
to debootstrap for RPM based distributions
./rpmbootstrap/rpm/rpmbootstrap.spec:Summary: PBSUMMARY
./rpmbootstrap/deb/control:Description: PBSUMMARY
./rpmbootstrap/pkg/pkginfo:NAME="PBSUMMARY"
14
Instantiation possible from distro-ver-arch, distro-ver, distro,
distro-family, distro-type, os
Similar approach for all other parameters (Cf: man pb.conf)
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
15. Commands
[cms|sbx]2build: Create tar files for the project under your CMS. Parameters are
packages to build if not using default list
build2pkg: Create packages for your running distribution
build2ssh: Send the tar files to a SSH host
pkg2ssh: Send the packages built to a SSH host
build2vm|ve|rm: Create packages in VMs/VEs/RMs, launching them if needed and
send those packages to a SSH host once built
launchvm: Launch one virtual machine
script2vm|ve|rm: Launch a VM/VE/RM if needed and executes a script on it
test2vm|ve|rm: Test inside a VM/VE/RM
newvm|ve: Create a new VM/VE/RM
setupvm|ve|rm: Setup a VM/VE/RM for pb usage
newver: Create a new version of the project derived from the current one
newproj: Create a new project and a template set of configuration files under
pbconf
15
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
16. The Big picture
Packagers
newver / newproj
Packaging
newvm|ve / setupvm|ve|rm
build2vm|ve|rm
cms|sbx2build
Project
16
Developers
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
Building
build2pkg
build2ssh
Packages
pkg2ssh
17. Multi platform packaging cycle
VMs/VEs/RMs
Developers
UUWL
Packages
17
VM/VE/RM
Packagers
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
18. Q & A up to now ?
18
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
20. Special announce for LCA 2014 !
20
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
21. More Special announce for LCA 2014 !
I hate time release delivery
Of course, 0.12.3 has bugs ;-)
Lack of time to test thoroughly
So I hate time release delivery
So expect a 0.12.4 later this week, when more tests have been done !
So back to feature set release in fact
21
Did I say I hate time release delivery ?
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
22. To be done
LSB chroot support. (or not)
LXC support (which one ?)
Support for libvirt, virsh, openstack API
Other CMS (Bazaar, …) only when/if needed
Other VMs (VMWare, Xen, ...) only when/if needed
Multiple delivery means
22
Look at interactions with Buildbot
Add non-interactive signature support for .deb
Config-Model for configuration file management
REST API and Web Interface would be nice. Any contributor ?
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
23. Learn Project-Builder.org
Start with the Lab (63 pages)
Use man (pb, pb.conf + 8
ProjectBuilder::* man pages)
Use the mailing-list pb-announce
and pb-devel at
http://www.mondorescue.org/sympa
Use examples from
23
http://trac.project-builder.org/browser/projects/
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
24. Web Resources
Project-Builder Web site / Trac / Wiki:
●
http://www.project-builder.org
●
http://trac.project-builder.org
Projects using project-builder.org:
●
http://www.project-builder.org ( of course :-)
●
http://www.mondorescue.org
24
●
Http://www.linuxcoe.org
●
http://www.fossology.org
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
25. Related tools
Project-Builder is mostly suited for upstream projects wanting to package their
applications
Distributions provide each their build tools
● SuSE: Open Build Service (Multi distro, Web based, BaaS)
● Fedora (Koji)
● Mandriva/Mageia (Youri, mdvsys, mgarepo...)
Other complementary tools:
● Buildbot
● KVM/QEMU
● rpmbootstrap, rinse, mock
● Parallel::ForkManager, DBI, DB::SQLite, File::MimeInfo, Mail::Sendmail
25
Project-Builder.org is one of the tool of the vcs-pkg.org initiative.
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
26. Contact - Thanks
Bruno.Cornec@hp.com
(Open Source and Linux Technology
Architect at the HP/Intel Solution Center)
http://www.hp.com/linux
http://opensource.hp.com
Thanks goes to:
26
Linus Torvalds, Richard Stallman, Eric
Raymond, Nat Makarevitch, René
Cougnenc, Eric Dumas, Rémy Card,
Bdale Garbee, Bryan Gartner, Craig
Lamparter, Lee Mayes, Gallig Renaud,
Andree Leidenfrost, Phil Robb, Bob
Gobeille, Martin Michlmayr among
others, for their work and devotion to
the Open Source Software cause... and
my family for their patience :-)
© Copyright 2012 Hewlett-Packard Development Company, L.P. - Licensed under CC-by-SA 3.0
”Changes are never easy to make.
There is comfort and safety in
tradition, but change must come, no
matter how painful or expensive it
may be.”
Bill Hewlett