Session ID: SFO17-407 Session Name: Collaboras involvement in KernelCI - SFO17-407 Speaker: Guillaume Tucker Track: ★ Session Summary ★ This talk will address Collabora's contribution to KernelCI, specifically investigating automated bisection. --------------------------------------------------- ★ Resources ★ Event Page: http://connect.linaro.org/resource/sfo17/sfo17-407/ Presentation: Video: --------------------------------------------------- ★ Event Details ★ Linaro Connect San Francisco 2017 (SFO17) 25-29 September 2017 Hyatt Regency San Francisco Airport --------------------------------------------------- Keyword: http://www.linaro.org http://connect.linaro.org --------------------------------------------------- Follow us on Social Media https://www.facebook.com/LinaroOrg https://twitter.com/linaroorg https://www.youtube.com/user/linaroorg?sub_confirmation=1 https://www.linkedin.com/company/1026961

1. kernelci.org
Project overview and Collabora’s contribution
Guillaume Tucker
gtucker@collabora.com
28 Sep 2017

2. KernelCI: the project
●
Build and boot Linux kernel revisions continuously
– Detect and report failures almost in real time
●
Community
– Launched by Linaro, now has various contributors
●
Mostly targeting embedded ARM-based platforms
– Running mainline trees, not vendor kernels
●
Simple tests that catch fundamental issues early
2

3. KernelCI: the infrastructure
Mongo
database
KernelCI
back-end
web API
Jenkins
LAVA labs
web API
KernelCI
front-end
web UI
Linux source
git
web UI
1
2
3
4
5
6
3

4. KernelCI: the test cycle
1. Some code lands in the Linux git repository
2. Jenkins triggers builds for various configurations
3. Jenkins triggers LAVA boots with the built kernels
4. Results are pushed to the KernelCI back-end
5. Results are available via the front-end and emails
6. Developers can fix issues and send new patches...
4

5. Labs
Kevin Hilman’sSjoerd’s5
Cambridge

6. KernelCI: the labs
●
Many various labs connected to KernelCI
– Linaro, Collabora, BayLibre, Pengutronix...
– Individuals: Kevin Hilman, Mark Brown…
●
Linaro Automated Validation Architecture - LAVA
– https://validation.linaro.org/static/docs/v2/index.html
●
Not all labs use LAVA
– target is to align everything with LAVA v2
– some quirky devices are hard to automate
6

7. ●
LAVA lab
– Up to 1000 KernelCI boot tests per day, ~20 boards
– Sponsored VMs to run builds in Jenkins
●
Enabled new device types (boards) in KernelCI
●
Various contributions to LAVA
– Bug fixes, IPv6, flashing images
●
KernelCI core improvements
– LAVA v2 callback support, automatic bisection
Extra information 7
Collabora’s contribution

8. Automatic Bisection
●
Some branches get merged with many commits
●
When a boot fails, only a range of commits is known
●
Goal: automatically isolate the breaking commit
●
Challenges:
– Avoiding false positives
– Triggering relevant bisections (avoid duplicates...)
– Deploying into production
– Providing useful results
8

9. Automatic Bisection – current status
●
Currently on staging, getting deployed:
– Jenkins job to run a bisection using LAVA v2
– Logic in the back-end to determine bisection ranges
– Back-end triggers bisections after each boot report
●
Things to do:
– Store bisection results, not just boot results
– Add checks to not run unwanted bisections
– Test and bring to maturity
9

10. Automatic Bisection – what next?
●
Make bisection results visible on the front-end
●
Send email notifications with bisection results
●
Include bisection of failing tests, not only boots
●
Consolidate back-end API around regressions
●
Add option to bisect against a different tree
– For example, failure in linux-next against master
●
Any other thoughts?
10

11. Other plans and controversial ideas
●
Add more tests in addition to kselftest
– See https://staging.kernelci.org/test/
– hackbench, igt maybe for DRM/KMS coverage?
●
New Android “modular kernel” trees?
●
Add support for depthcharge (Chrome OS)
●
Improve quality so people trust the boot results more
– align boards’ bootloader configs
●
Add support for multiple compilers
11

12. Thank you!
Copyright @ 2017 Collabora Limited. Some rights reserved.