SlideShare une entreprise Scribd logo
1  sur  49
Télécharger pour lire hors ligne
Yocto Project | The Linux Foundation
Marco Cavallini, KOAN
https://koansoftware.com
1. Strenghten your Yocto deployments with
Autobuilder2 CI tool
2 Yocto Project | The Linux Foundation
Who am I
• Founded KOAN on 1996
• Working with software for industrial automation until 1999
• Linux embedded devloper since 2000
• Openembedded board member since 2009
• Yocto Project participant since 2010
• Yocto Project Advocate since 2012
•
• Software development and consulting
• BSP creation
• Device driver kernel development
• Open embedded and Yocto Project training
3 Yocto Project | The Linux Foundation
Agenda
• What is a Continuous Integration (CI) system
• Autobuilder2 History
• Buildbot, the foundations
• Buildbot mechanics
• Buildbot installation
• Autobuilder2 installation
• Autobuilder2 configuration
• Autobuilder2 usage (as-is)
• Autobuilder2 customization
• Autobuilder2 usage for CI on a single machine
4 Yocto Project | The Linux Foundation
What is Autobuilder
• Autobuilder is a project based on Buildbot
• Buildbot is a Python open-source application used to
build, test, and release a wide variety of software.
• Autobuilder and Buildbot are licensed under GPLv2
5 Yocto Project | The Linux Foundation
CI overview
‘master’
CONTROLLER
• Typical CI on a single machine
‘slave’
WORKER
‘slave’
WORKER
‘slave’
WORKER
6 Yocto Project | The Linux Foundation
CI overview
‘master’
CONTROLLER
• Typical CI on a distributed system
‘slave’
WORKER
‘slave’
WORKER
‘slave’
WORKER
7 Yocto Project | The Linux Foundation
Our goal
‘master’
CONTROLLER
• Speed Yocto builds by populating premirrors with
Autobuilder2
‘slave’
WORKER
SSTATE
image
DL_DIR bitbake
8 Yocto Project | The Linux Foundation
Autobuilder overview
yocto-autobuilder2 yocto-autobuilder-helper
9 Yocto Project | The Linux Foundation
Autobuilder history *
• Creation of Autobuilder
• The autobuilder started life as something OpenedHand used
for testing Poky linux.
• Yocto-autobuilder
• It became "yocto-autobuilder" under Beth's stewardship in
December 2012 when it was totally re-implemented.
• Autobuilder2
• In Feburary 2018 it was rewritten again, in particular to move
from the long obsolete "buildbot eight" codebase to the
"buildbot nine" one but also to fix many long running issues
and get back to using an upstream codebase.
* Thanks to Richard Purdie who provided these information
10 Yocto Project | The Linux Foundation
People behind Autobuilder *
• Project Autobuilder
• Richard Purdie, Elizabeth Flanagan, Joshua Lock as well as
contributions from Tracy Graydon, Anibal Limón and Bill
Randle.
• Project Autobuilder2
• Richard Purdie and Joshua Lock.
• Michael Halstead is the project sysadmin who maintains the
infrastructure it all runs on top of.
* Thanks to Richard Purdie who provided these information
11 Yocto Project | The Linux Foundation
Buildbot
12 Yocto Project | The Linux Foundation
Buildbot basics
• Buildbot is a job scheduling system
• it queues jobs, executes the jobs when the required
resources are available, and reports the results
13 Yocto Project | The Linux Foundation
Buildbot basics
• Workers are typically run on separate machines
14 Yocto Project | The Linux Foundation
Buildbot basics
15 Yocto Project | The Linux Foundation
What happens inside the master
builder n
worker 1 worker nworker 2 . . .
↓cmds results↑results
VCS
repos
change source
poll
changes
scheduler
triggers
direct commands
Buildfactory
- step 1
- step 2
.
.
.
reporter
status/results
Build master
checkout
builder 2
builder 1
. . .
starts builder
scheduled events
* artwork by Mauro Salvini
16 Yocto Project | The Linux Foundation
Buildbot installation
• On a native system
• Probably the fastest solution
• In a Python sandbox
• Isolates it from the host system
• Using pip
• In a Docker container
• Isolates it from the host system
https://docs.buildbot.net/2.4.0/full.html
17 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [1/3]
• Create a sandbox
• Install master
• Install worker
mkdir abot-sandbox
cd abot-sandbox
python3 -m venv sandbox
source sandbox/bin/activate
mkdir abot-sandbox
cd abot-sandbox
python3 -m venv sandbox
source sandbox/bin/activate
pip install --upgrade pip
pip install 'buildbot[bundle]'
pip install --upgrade pip
pip install 'buildbot[bundle]'
pip install --upgrade pip
pip install buildbot-worker
pip install --upgrade pip
pip install buildbot-worker
18 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [2/3]
• Create the master
• Create the worker
buildbot create-master master
mv master/master.cfg.sample master/master.cfg
buildbot create-master master
mv master/master.cfg.sample master/master.cfg
pip install setuptools-trial
buildbot-worker create-worker worker localhost example-worker pass
pip install setuptools-trial
buildbot-worker create-worker worker localhost example-worker pass
19 Yocto Project | The Linux Foundation
Buildbot installation (in a Python sandbox) [3/3]
• Content of the sandbox
(sandbox) koan@amonra:~/abot-sandbox$ tree -L 2
.
├── master
│   ├── buildbot.tac
│   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*)
│   └── state.sqlite
├── sandbox
│   ├── bin
│   ├── include
│   ├── lib
│   ├── lib64 -> lib
│   ├── pip-selfcheck.json
│   ├── pyvenv.cfg
│   └── share
└── worker
├── buildbot.tac → workername = 'example-worker' (*)
└── info
(sandbox) koan@amonra:~/abot-sandbox$ tree -L 2
.
├── master
│   ├── buildbot.tac
│   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*)
│   └── state.sqlite
├── sandbox
│   ├── bin
│   ├── include
│   ├── lib
│   ├── lib64 -> lib
│   ├── pip-selfcheck.json
│   ├── pyvenv.cfg
│   └── share
└── worker
├── buildbot.tac → workername = 'example-worker' (*)
└── info
ftp://ftp.koansoftware.com/public/opensource/buildbot/(*)
20 Yocto Project | The Linux Foundation
Buildbot execution
• Execution of master
• Control of the build system (using your browser)
buildbot start masterbuildbot start master
http://localhost:8010/http://localhost:8010/
21 Yocto Project | The Linux Foundation
Buildbot
• To be continued in a dedicated session…
• Now let’s have a look at Autobuilder
22 Yocto Project | The Linux Foundation
Autobuilder2
Ab2
Ab2
23 Yocto Project | The Linux Foundation
Buildbot vs. Autobuilder2 lexicon Ab2
Buildbot Autobuilder2
Master Controller
Worker Worker
24 Yocto Project | The Linux Foundation
Autobuilder2 installation
• After you created the sandbox
• Create the master and worker directories
• yocto-controller is the directory for master
• --umask sets the proper permissions
• yocto-worker is the directory for worker
• localhost is the network address of the master
• example-worker is the name of the worker
• pass is the password (master.cfg)
Ab2
buildbot create-master -r yocto-controller
buildbot-worker create-worker -r /
--umask=0o22 yocto-worker localhost example-worker pass
buildbot create-master -r yocto-controller
buildbot-worker create-worker -r /
--umask=0o22 yocto-worker localhost example-worker pass
25 Yocto Project | The Linux Foundation
Autobuilder2 installation
• Clone yocto-autobuilder2
• yoctoabb is the mandatory Autobuilder2 directory name
• Clone yocto-autobuilder-helper
cd yocto-controller
git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
ln -rs yoctoabb/master.cfg master.cfg
cd yocto-controller
git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb
ln -rs yoctoabb/master.cfg master.cfg
cd ..
git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
cd ..
git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper
Ab2
26 Yocto Project | The Linux Foundation
Autobuilder2 tree
• Content of the Autobuilder2 sandbox
(sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3
.
├── autobuilder
├── git
├── sandbox
├── yocto-autobuilder-helper
├── yocto-controller
│   └── yoctoabb
│   └── master.cfg
└── yocto-worker
└── buildbot.tac
(sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3
.
├── autobuilder
├── git
├── sandbox
├── yocto-autobuilder-helper
├── yocto-controller
│   └── yoctoabb
│   └── master.cfg
└── yocto-worker
└── buildbot.tac
Ab2
* Simplified tree list
27 Yocto Project | The Linux Foundation
Autobuilder2 installation
• Complete installation instructions
Ab2
http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder2/tree/README-Guide.md
html
ftp://ftp.koansoftware.com/public/talks/YoctoSummit-2019/au
tobuilder2/Autobuilder2-README-Guide.pdf
28 Yocto Project | The Linux Foundation
Autobuilder2
Autobuilder default configuration
29 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• Ab2 default configuration
• The default configuration of Ab2 uses a lot of workers to
generate images for several MACHINES
Ab2
30 Yocto Project | The Linux Foundation
Autobuilder2 official website
https://autobuilder.yoctoproject.org/typhoon/
31 Yocto Project | The Linux Foundation
Autobuilder2 website navigation
32 Yocto Project | The Linux Foundation
Autobuilder2 website navigation
33 Yocto Project | The Linux Foundation
Autobuilder2 - builds
34 Yocto Project | The Linux Foundation
Autobuilder2 - workers
35 Yocto Project | The Linux Foundation
Autobuilder2 – worker details
36 Yocto Project | The Linux Foundation
Autobuilder2 – build details
37 Yocto Project | The Linux Foundation
Autobuilder2 – build details
38 Yocto Project | The Linux Foundation
Autobuilder2 – build details
39 Yocto Project | The Linux Foundation
Autobuilder2 – build details
40 Yocto Project | The Linux Foundation
Autobuilder2
Autobuilder custom ‘lighter’
configuration
41 Yocto Project | The Linux Foundation
Autobuilder2 reduced configuration
• Reduce complexity
• The goal is to setup a configuration for an image for a single
MACHINE only
• This will help yo strenghten the deployments thanks to
recurring builds, tipically nightly
Ab2
42 Yocto Project | The Linux Foundation
Autobuilder2 reduced configuration
• Files to be modified
.
├── yocto-autobuilder-helper
│   └── config.json
├── yocto-controller
│   └── yoctoabb
│   ├── builders.py
│   ├── config.py
│   ├── master.cfg
│   └── schedulers.py
└── yocto-worker
└── buildbot.tac
.
├── yocto-autobuilder-helper
│   └── config.json
├── yocto-controller
│   └── yoctoabb
│   ├── builders.py
│   ├── config.py
│   ├── master.cfg
│   └── schedulers.py
└── yocto-worker
└── buildbot.tac
Ab2
43 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• In yocto-autobuilder-helper
• Edit yocto-autobuilder-helper/config.json
• In yocto-controller
• Edit yocto-controller/yoctoabb/master.cfg
"BASE_HOMEDIR" : "/home/koan/ab2-sandbox",
"BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder",
"BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads",
"BASE_HOMEDIR" : "/home/koan/ab2-sandbox",
"BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder",
"BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads",
c['title'] = "KOAN lite Yocto Autobuilder"
c['titleURL'] = "http://localhost:8010/"
c['buildbotURL'] = "http://localhost:8010/"
c['title'] = "KOAN lite Yocto Autobuilder"
c['titleURL'] = "http://localhost:8010/"
c['buildbotURL'] = "http://localhost:8010/"
Ab2
44 Yocto Project | The Linux Foundation
Autobuilder2 configuration
• In yocto-controller (again)
• Edit yocto-controller/yoctoabb/config.py
• Specify the helper directory
workers_koan = ["example-worker"]
workers = workers_koan
all_workers = workers,
workers_koan = ["example-worker"]
workers = workers_koan
all_workers = workers,
sharedrepodir = "/home/koan/ab2-sandbox/repos"
publish_dest = "/home/koan/ab2-sandbox/pub"
sharedrepodir = "/home/koan/ab2-sandbox/repos"
publish_dest = "/home/koan/ab2-sandbox/pub"
Ab2
repos = {
"yocto-autobuilder-helper":
["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper",
"master"],
repos = {
"yocto-autobuilder-helper":
["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper",
"master"],
45 Yocto Project | The Linux Foundation
Autobuilder2 ‘lite’ customized
http://localhost:8010/
46 Yocto Project | The Linux Foundation
Autobuilder2 ‘lite’ customized
47 Yocto Project | The Linux Foundation
Autobuilder2 to speed up Yocto build
• Share the artefacts with Yocto
• Edit local.conf
• Share downloads
• Share SSTATE
PREMIRRORS_prepend = "
git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
"
PREMIRRORS_prepend = "
git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n 
"
Ab2
SSTATE_MIRRORS = "file://.* 
file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
SSTATE_MIRRORS = "file://.* 
file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
48 Yocto Project | The Linux Foundation
Autobuilder2 to speed up Yocto build
‘master’
CONTROLLER
‘slave’
WORKER
SSTATE
image
DL_DIR bitbake
Yocto Project | The Linux Foundation
Questions?
https://koansoftware.comhttps://yoctoproject.org

Contenu connexe

Tendances

The Yocto Project
The Yocto ProjectThe Yocto Project
The Yocto Projectrossburton
 
yocto_scale_handout-with-notes
yocto_scale_handout-with-notesyocto_scale_handout-with-notes
yocto_scale_handout-with-notesSteve Arnold
 
Build your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto projectBuild your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto projectYen-Chin Lee
 
Yocto and IoT - a retrospective
Yocto and IoT - a retrospectiveYocto and IoT - a retrospective
Yocto and IoT - a retrospectiveOpen-RnD
 
Yocto: Treinamento em Português
Yocto: Treinamento em PortuguêsYocto: Treinamento em Português
Yocto: Treinamento em PortuguêsOtavio Salvador
 
Eclipse IDE Yocto Plugin
Eclipse IDE Yocto PluginEclipse IDE Yocto Plugin
Eclipse IDE Yocto Plugincudma
 
Yocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution MakerYocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution MakerSherif Mousa
 
Debugging embedded devices using GDB
Debugging embedded devices using GDBDebugging embedded devices using GDB
Debugging embedded devices using GDBChris Simmonds
 
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...Anne Nicolas
 
Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?Chris Simmonds
 
Introduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux FrameworkIntroduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux FrameworkBenjamin Zores
 
Embedded Linux primer
Embedded Linux primerEmbedded Linux primer
Embedded Linux primerDrew Fustini
 
The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)Chris Simmonds
 
Reducing the boot time of Linux devices
Reducing the boot time of Linux devicesReducing the boot time of Linux devices
Reducing the boot time of Linux devicesChris Simmonds
 
Reducing boot time in embedded Linux
Reducing boot time in embedded LinuxReducing boot time in embedded Linux
Reducing boot time in embedded LinuxChris Simmonds
 
Droidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded SystemsDroidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded SystemsBenjamin Zores
 
Building Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto projectBuilding Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto projecttwcoimbatore
 
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processorUplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processorSatya Harish
 

Tendances (20)

The Yocto Project
The Yocto ProjectThe Yocto Project
The Yocto Project
 
Yocto project
Yocto projectYocto project
Yocto project
 
yocto_scale_handout-with-notes
yocto_scale_handout-with-notesyocto_scale_handout-with-notes
yocto_scale_handout-with-notes
 
Build your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto projectBuild your own embedded linux distributions by yocto project
Build your own embedded linux distributions by yocto project
 
Yocto and IoT - a retrospective
Yocto and IoT - a retrospectiveYocto and IoT - a retrospective
Yocto and IoT - a retrospective
 
Yocto: Treinamento em Português
Yocto: Treinamento em PortuguêsYocto: Treinamento em Português
Yocto: Treinamento em Português
 
Eclipse IDE Yocto Plugin
Eclipse IDE Yocto PluginEclipse IDE Yocto Plugin
Eclipse IDE Yocto Plugin
 
Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012Yocto Project - OSCON 7-17-2012
Yocto Project - OSCON 7-17-2012
 
Yocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution MakerYocto - Embedded Linux Distribution Maker
Yocto - Embedded Linux Distribution Maker
 
Debugging embedded devices using GDB
Debugging embedded devices using GDBDebugging embedded devices using GDB
Debugging embedded devices using GDB
 
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
Embedded Recipes 2017 - Introduction to Yocto Project/OpenEmbedded - Mylène J...
 
Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?Debian or Yocto Project? Which is the best for your Embedded Linux project?
Debian or Yocto Project? Which is the best for your Embedded Linux project?
 
Introduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux FrameworkIntroduction to OpenBricks: an Embedded Linux Framework
Introduction to OpenBricks: an Embedded Linux Framework
 
Embedded Linux primer
Embedded Linux primerEmbedded Linux primer
Embedded Linux primer
 
The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)The end of embedded Linux (as we know it)
The end of embedded Linux (as we know it)
 
Reducing the boot time of Linux devices
Reducing the boot time of Linux devicesReducing the boot time of Linux devices
Reducing the boot time of Linux devices
 
Reducing boot time in embedded Linux
Reducing boot time in embedded LinuxReducing boot time in embedded Linux
Reducing boot time in embedded Linux
 
Droidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded SystemsDroidcon 2013 france - The Growth of Android in Embedded Systems
Droidcon 2013 france - The Growth of Android in Embedded Systems
 
Building Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto projectBuilding Emmbedded Linux with Yocto project
Building Emmbedded Linux with Yocto project
 
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processorUplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
UplinQ - ubuntu linux on the qualcomm® snapdragon™ 600 processor
 

Similaire à Autobuilder2 Yocto Project Summit Lyon 2019

Yocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-onYocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-onTrevor Woerner
 
An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018ICS
 
Yocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-OnYocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-OnTrevor Woerner
 
Stefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto ProjectStefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto Projectlinuxlab_conf
 
IoT Prototyping using BBB and Debian
IoT Prototyping using BBB and DebianIoT Prototyping using BBB and Debian
IoT Prototyping using BBB and DebianMender.io
 
IoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSesIoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSesMender.io
 
FOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdfFOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdfssuser30e7d2
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems Mender.io
 
[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded FrameworkICS
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building ToolsAkihiro Suda
 
Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018Mender.io
 
Introduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systemsIntroduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systemsNaohiko Shimizu
 
LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017Dieter Reuter
 
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TXLinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TXDieter Reuter
 
Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018Mender.io
 
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily JobLuca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Joblinuxlab_conf
 
Creating new Tizen profiles using the Yocto Project
Creating new Tizen profiles  using the Yocto ProjectCreating new Tizen profiles  using the Yocto Project
Creating new Tizen profiles using the Yocto ProjectLeon Anavi
 
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...LF Events
 

Similaire à Autobuilder2 Yocto Project Summit Lyon 2019 (20)

Yocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-onYocto Project Kernel Lab hands-on
Yocto Project Kernel Lab hands-on
 
An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018An Introduction to the Yocto Embedded Framework 2018
An Introduction to the Yocto Embedded Framework 2018
 
Yocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-OnYocto Project Kernel Lab, Hands-On
Yocto Project Kernel Lab, Hands-On
 
Stefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto ProjectStefano Cordibella - An introduction to Yocto Project
Stefano Cordibella - An introduction to Yocto Project
 
IoT Prototyping using BBB and Debian
IoT Prototyping using BBB and DebianIoT Prototyping using BBB and Debian
IoT Prototyping using BBB and Debian
 
IoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSesIoT: Contrasting Yocto/Buildroot to binary OSes
IoT: Contrasting Yocto/Buildroot to binary OSes
 
FOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdfFOSDEM_2019_Buildroot_RISCV.pdf
FOSDEM_2019_Buildroot_RISCV.pdf
 
Building Embedded Linux UDOONEO
Building Embedded Linux UDOONEOBuilding Embedded Linux UDOONEO
Building Embedded Linux UDOONEO
 
Embedded linux build systems
Embedded linux build systems  Embedded linux build systems
Embedded linux build systems
 
[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework[Webinar] An Introduction to the Yocto Embedded Framework
[Webinar] An Introduction to the Yocto Embedded Framework
 
Comparing Next-Generation Container Image Building Tools
 Comparing Next-Generation Container Image Building Tools Comparing Next-Generation Container Image Building Tools
Comparing Next-Generation Container Image Building Tools
 
Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018Embedded Linux Build Systems - Texas Linux Fest 2018
Embedded Linux Build Systems - Texas Linux Fest 2018
 
Introduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systemsIntroduction to the rapid prototyping with python and linux for embedded systems
Introduction to the rapid prototyping with python and linux for embedded systems
 
LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017LinuxKit and Moby, News from DockerCon 2017
LinuxKit and Moby, News from DockerCon 2017
 
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TXLinuxKit and Moby, news from DockerCon 2017 - Austin,TX
LinuxKit and Moby, news from DockerCon 2017 - Austin,TX
 
Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018Why the yocto project for my io t project elc_edinburgh_2018
Why the yocto project for my io t project elc_edinburgh_2018
 
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily JobLuca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
Luca Ceresoli - Buildroot vs Yocto: Differences for Your Daily Job
 
Creating new Tizen profiles using the Yocto Project
Creating new Tizen profiles  using the Yocto ProjectCreating new Tizen profiles  using the Yocto Project
Creating new Tizen profiles using the Yocto Project
 
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
Generating a Reproducible and Maintainable Embedded Linux Environment with Po...
 
Firebird on Linux
Firebird on LinuxFirebird on Linux
Firebird on Linux
 

Dernier

Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfBrain Inventory
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilVICTOR MAESTRE RAMIREZ
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 

Dernier (20)

Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 
Why Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdfWhy Choose Brain Inventory For Ecommerce Development.pdf
Why Choose Brain Inventory For Ecommerce Development.pdf
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Generative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-CouncilGenerative AI for Cybersecurity - EC-Council
Generative AI for Cybersecurity - EC-Council
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 

Autobuilder2 Yocto Project Summit Lyon 2019

  • 1. Yocto Project | The Linux Foundation Marco Cavallini, KOAN https://koansoftware.com 1. Strenghten your Yocto deployments with Autobuilder2 CI tool
  • 2. 2 Yocto Project | The Linux Foundation Who am I • Founded KOAN on 1996 • Working with software for industrial automation until 1999 • Linux embedded devloper since 2000 • Openembedded board member since 2009 • Yocto Project participant since 2010 • Yocto Project Advocate since 2012 • • Software development and consulting • BSP creation • Device driver kernel development • Open embedded and Yocto Project training
  • 3. 3 Yocto Project | The Linux Foundation Agenda • What is a Continuous Integration (CI) system • Autobuilder2 History • Buildbot, the foundations • Buildbot mechanics • Buildbot installation • Autobuilder2 installation • Autobuilder2 configuration • Autobuilder2 usage (as-is) • Autobuilder2 customization • Autobuilder2 usage for CI on a single machine
  • 4. 4 Yocto Project | The Linux Foundation What is Autobuilder • Autobuilder is a project based on Buildbot • Buildbot is a Python open-source application used to build, test, and release a wide variety of software. • Autobuilder and Buildbot are licensed under GPLv2
  • 5. 5 Yocto Project | The Linux Foundation CI overview ‘master’ CONTROLLER • Typical CI on a single machine ‘slave’ WORKER ‘slave’ WORKER ‘slave’ WORKER
  • 6. 6 Yocto Project | The Linux Foundation CI overview ‘master’ CONTROLLER • Typical CI on a distributed system ‘slave’ WORKER ‘slave’ WORKER ‘slave’ WORKER
  • 7. 7 Yocto Project | The Linux Foundation Our goal ‘master’ CONTROLLER • Speed Yocto builds by populating premirrors with Autobuilder2 ‘slave’ WORKER SSTATE image DL_DIR bitbake
  • 8. 8 Yocto Project | The Linux Foundation Autobuilder overview yocto-autobuilder2 yocto-autobuilder-helper
  • 9. 9 Yocto Project | The Linux Foundation Autobuilder history * • Creation of Autobuilder • The autobuilder started life as something OpenedHand used for testing Poky linux. • Yocto-autobuilder • It became "yocto-autobuilder" under Beth's stewardship in December 2012 when it was totally re-implemented. • Autobuilder2 • In Feburary 2018 it was rewritten again, in particular to move from the long obsolete "buildbot eight" codebase to the "buildbot nine" one but also to fix many long running issues and get back to using an upstream codebase. * Thanks to Richard Purdie who provided these information
  • 10. 10 Yocto Project | The Linux Foundation People behind Autobuilder * • Project Autobuilder • Richard Purdie, Elizabeth Flanagan, Joshua Lock as well as contributions from Tracy Graydon, Anibal Limón and Bill Randle. • Project Autobuilder2 • Richard Purdie and Joshua Lock. • Michael Halstead is the project sysadmin who maintains the infrastructure it all runs on top of. * Thanks to Richard Purdie who provided these information
  • 11. 11 Yocto Project | The Linux Foundation Buildbot
  • 12. 12 Yocto Project | The Linux Foundation Buildbot basics • Buildbot is a job scheduling system • it queues jobs, executes the jobs when the required resources are available, and reports the results
  • 13. 13 Yocto Project | The Linux Foundation Buildbot basics • Workers are typically run on separate machines
  • 14. 14 Yocto Project | The Linux Foundation Buildbot basics
  • 15. 15 Yocto Project | The Linux Foundation What happens inside the master builder n worker 1 worker nworker 2 . . . ↓cmds results↑results VCS repos change source poll changes scheduler triggers direct commands Buildfactory - step 1 - step 2 . . . reporter status/results Build master checkout builder 2 builder 1 . . . starts builder scheduled events * artwork by Mauro Salvini
  • 16. 16 Yocto Project | The Linux Foundation Buildbot installation • On a native system • Probably the fastest solution • In a Python sandbox • Isolates it from the host system • Using pip • In a Docker container • Isolates it from the host system https://docs.buildbot.net/2.4.0/full.html
  • 17. 17 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [1/3] • Create a sandbox • Install master • Install worker mkdir abot-sandbox cd abot-sandbox python3 -m venv sandbox source sandbox/bin/activate mkdir abot-sandbox cd abot-sandbox python3 -m venv sandbox source sandbox/bin/activate pip install --upgrade pip pip install 'buildbot[bundle]' pip install --upgrade pip pip install 'buildbot[bundle]' pip install --upgrade pip pip install buildbot-worker pip install --upgrade pip pip install buildbot-worker
  • 18. 18 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [2/3] • Create the master • Create the worker buildbot create-master master mv master/master.cfg.sample master/master.cfg buildbot create-master master mv master/master.cfg.sample master/master.cfg pip install setuptools-trial buildbot-worker create-worker worker localhost example-worker pass pip install setuptools-trial buildbot-worker create-worker worker localhost example-worker pass
  • 19. 19 Yocto Project | The Linux Foundation Buildbot installation (in a Python sandbox) [3/3] • Content of the sandbox (sandbox) koan@amonra:~/abot-sandbox$ tree -L 2 . ├── master │   ├── buildbot.tac │   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*) │   └── state.sqlite ├── sandbox │   ├── bin │   ├── include │   ├── lib │   ├── lib64 -> lib │   ├── pip-selfcheck.json │   ├── pyvenv.cfg │   └── share └── worker ├── buildbot.tac → workername = 'example-worker' (*) └── info (sandbox) koan@amonra:~/abot-sandbox$ tree -L 2 . ├── master │   ├── buildbot.tac │   ├── master.cfg → c['workers'] = [worker.Worker("example-worker", "pass")] (*) │   └── state.sqlite ├── sandbox │   ├── bin │   ├── include │   ├── lib │   ├── lib64 -> lib │   ├── pip-selfcheck.json │   ├── pyvenv.cfg │   └── share └── worker ├── buildbot.tac → workername = 'example-worker' (*) └── info ftp://ftp.koansoftware.com/public/opensource/buildbot/(*)
  • 20. 20 Yocto Project | The Linux Foundation Buildbot execution • Execution of master • Control of the build system (using your browser) buildbot start masterbuildbot start master http://localhost:8010/http://localhost:8010/
  • 21. 21 Yocto Project | The Linux Foundation Buildbot • To be continued in a dedicated session… • Now let’s have a look at Autobuilder
  • 22. 22 Yocto Project | The Linux Foundation Autobuilder2 Ab2 Ab2
  • 23. 23 Yocto Project | The Linux Foundation Buildbot vs. Autobuilder2 lexicon Ab2 Buildbot Autobuilder2 Master Controller Worker Worker
  • 24. 24 Yocto Project | The Linux Foundation Autobuilder2 installation • After you created the sandbox • Create the master and worker directories • yocto-controller is the directory for master • --umask sets the proper permissions • yocto-worker is the directory for worker • localhost is the network address of the master • example-worker is the name of the worker • pass is the password (master.cfg) Ab2 buildbot create-master -r yocto-controller buildbot-worker create-worker -r / --umask=0o22 yocto-worker localhost example-worker pass buildbot create-master -r yocto-controller buildbot-worker create-worker -r / --umask=0o22 yocto-worker localhost example-worker pass
  • 25. 25 Yocto Project | The Linux Foundation Autobuilder2 installation • Clone yocto-autobuilder2 • yoctoabb is the mandatory Autobuilder2 directory name • Clone yocto-autobuilder-helper cd yocto-controller git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb ln -rs yoctoabb/master.cfg master.cfg cd yocto-controller git clone https://git.yoctoproject.org/git/yocto-autobuilder2 yoctoabb ln -rs yoctoabb/master.cfg master.cfg cd .. git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper cd .. git clone https://git.yoctoproject.org/git/yocto-autobuilder-helper Ab2
  • 26. 26 Yocto Project | The Linux Foundation Autobuilder2 tree • Content of the Autobuilder2 sandbox (sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3 . ├── autobuilder ├── git ├── sandbox ├── yocto-autobuilder-helper ├── yocto-controller │   └── yoctoabb │   └── master.cfg └── yocto-worker └── buildbot.tac (sandbox) koan@amonra:~/ab2-sandbox$ tree -L 3 . ├── autobuilder ├── git ├── sandbox ├── yocto-autobuilder-helper ├── yocto-controller │   └── yoctoabb │   └── master.cfg └── yocto-worker └── buildbot.tac Ab2 * Simplified tree list
  • 27. 27 Yocto Project | The Linux Foundation Autobuilder2 installation • Complete installation instructions Ab2 http://git.yoctoproject.org/cgit.cgi/yocto-autobuilder2/tree/README-Guide.md html ftp://ftp.koansoftware.com/public/talks/YoctoSummit-2019/au tobuilder2/Autobuilder2-README-Guide.pdf
  • 28. 28 Yocto Project | The Linux Foundation Autobuilder2 Autobuilder default configuration
  • 29. 29 Yocto Project | The Linux Foundation Autobuilder2 configuration • Ab2 default configuration • The default configuration of Ab2 uses a lot of workers to generate images for several MACHINES Ab2
  • 30. 30 Yocto Project | The Linux Foundation Autobuilder2 official website https://autobuilder.yoctoproject.org/typhoon/
  • 31. 31 Yocto Project | The Linux Foundation Autobuilder2 website navigation
  • 32. 32 Yocto Project | The Linux Foundation Autobuilder2 website navigation
  • 33. 33 Yocto Project | The Linux Foundation Autobuilder2 - builds
  • 34. 34 Yocto Project | The Linux Foundation Autobuilder2 - workers
  • 35. 35 Yocto Project | The Linux Foundation Autobuilder2 – worker details
  • 36. 36 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 37. 37 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 38. 38 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 39. 39 Yocto Project | The Linux Foundation Autobuilder2 – build details
  • 40. 40 Yocto Project | The Linux Foundation Autobuilder2 Autobuilder custom ‘lighter’ configuration
  • 41. 41 Yocto Project | The Linux Foundation Autobuilder2 reduced configuration • Reduce complexity • The goal is to setup a configuration for an image for a single MACHINE only • This will help yo strenghten the deployments thanks to recurring builds, tipically nightly Ab2
  • 42. 42 Yocto Project | The Linux Foundation Autobuilder2 reduced configuration • Files to be modified . ├── yocto-autobuilder-helper │   └── config.json ├── yocto-controller │   └── yoctoabb │   ├── builders.py │   ├── config.py │   ├── master.cfg │   └── schedulers.py └── yocto-worker └── buildbot.tac . ├── yocto-autobuilder-helper │   └── config.json ├── yocto-controller │   └── yoctoabb │   ├── builders.py │   ├── config.py │   ├── master.cfg │   └── schedulers.py └── yocto-worker └── buildbot.tac Ab2
  • 43. 43 Yocto Project | The Linux Foundation Autobuilder2 configuration • In yocto-autobuilder-helper • Edit yocto-autobuilder-helper/config.json • In yocto-controller • Edit yocto-controller/yoctoabb/master.cfg "BASE_HOMEDIR" : "/home/koan/ab2-sandbox", "BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder", "BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads", "BASE_HOMEDIR" : "/home/koan/ab2-sandbox", "BASE_SHAREDDIR" : "${BASE_HOMEDIR}/autobuilder", "BASE_PUBLISHDIR" : "${BASE_HOMEDIR}/downloads", c['title'] = "KOAN lite Yocto Autobuilder" c['titleURL'] = "http://localhost:8010/" c['buildbotURL'] = "http://localhost:8010/" c['title'] = "KOAN lite Yocto Autobuilder" c['titleURL'] = "http://localhost:8010/" c['buildbotURL'] = "http://localhost:8010/" Ab2
  • 44. 44 Yocto Project | The Linux Foundation Autobuilder2 configuration • In yocto-controller (again) • Edit yocto-controller/yoctoabb/config.py • Specify the helper directory workers_koan = ["example-worker"] workers = workers_koan all_workers = workers, workers_koan = ["example-worker"] workers = workers_koan all_workers = workers, sharedrepodir = "/home/koan/ab2-sandbox/repos" publish_dest = "/home/koan/ab2-sandbox/pub" sharedrepodir = "/home/koan/ab2-sandbox/repos" publish_dest = "/home/koan/ab2-sandbox/pub" Ab2 repos = { "yocto-autobuilder-helper": ["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper", "master"], repos = { "yocto-autobuilder-helper": ["file:///home/koan/ab2-sandbox/yocto-autobuilder-helper", "master"],
  • 45. 45 Yocto Project | The Linux Foundation Autobuilder2 ‘lite’ customized http://localhost:8010/
  • 46. 46 Yocto Project | The Linux Foundation Autobuilder2 ‘lite’ customized
  • 47. 47 Yocto Project | The Linux Foundation Autobuilder2 to speed up Yocto build • Share the artefacts with Yocto • Edit local.conf • Share downloads • Share SSTATE PREMIRRORS_prepend = " git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n " PREMIRRORS_prepend = " git://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n ftp://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n http://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n https://.*/.* file:///home/koan/ab2-sandbox/autobuilder/current_sources/ n " Ab2 SSTATE_MIRRORS = "file://.* file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH" SSTATE_MIRRORS = "file://.* file:///home/koan/ab2-sandbox/autobuilder/pub/sstate/PATH"
  • 48. 48 Yocto Project | The Linux Foundation Autobuilder2 to speed up Yocto build ‘master’ CONTROLLER ‘slave’ WORKER SSTATE image DL_DIR bitbake
  • 49. Yocto Project | The Linux Foundation Questions? https://koansoftware.comhttps://yoctoproject.org