Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Hydra: Continuous Integration and Testing for Demanding People: The Details
1. Hydra: Continuous Integration and Testing for
Demanding People: The Details
Sander van der Burg
Conference Compass
July 15, 2014
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
2. Continuous integration
We want to deliver and test software rapidly
We quickly want to see the impact of changes to the source
code and its dependencies.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
3. Hydra
Hydra: A Nix-based continuous integration server:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
4. Hydra
Hydra: A Nix-based continuous integration server:
Generic. Supports multiple programming language
environments and component technologies.
Deployment. Build and test environments are deployed
automatically and all dependencies are ensured to be present
and correct.
Variability. Multiple versions/variants of dependencies can
safely coexist.
Multi platform support. Builds can be easily delegated to
machines with a different operating system.
Scalability. Builds are transparently delegated to any machine
in a cluster capable of building it.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
5. Hydra
How to use Hydra to build or test stuff?
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
7. Hydra overview
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Components
Queue runner: Regularly checks what has changed and
what to build
Evaluator: Builds the jobs
Server: Web application making builds and test results
available
Nix: Package mananger responsible for the actual
builds and depedency management
9. The Nix package manager
A package manager borrowing concepts from purely functional
programming languages.
x = y ⇒ f (x) = f (y)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
10. Nix store
Main idea: store all packages
in isolation from each other:
/nix/store/40awryfqzp46m...
-disnix-0.3
Paths contain a 160-bit
cryptographic hash of all
inputs used to build the
package:
Sources
Libraries
Compilers
Build scripts
. . .
/nix/store
40awryfqzp...-disnix-0.3
bin
disnix-env
disnix-manifest
disnix-service
kjlv4klmra...-getopt-1.1.4
bin
getopt
am13rq9ka...-dbus-glib-0.102
lib
libdbus-glib-1.so.2
94n64qy99...-glibc-2.19
lib
libc.so.6
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
11. Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
12. Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Environments
Expression defines a function that composes an
environment in which a build is executed
Nearly any type of build can be performed inside it,
e.g. C/C++, Java, Perl, Python...
We can also run tests inside these environments
The buildInputs parameters are used to configure all
settings to make a build find its dependencies, e.g.
setting PATH, PYTHONPATH, CLASSPATH ...
There are also function abstractions for different kinds
of packages
If no buildCommand is given, it executes the default
GNU Autotools build procedure: ./configure;
make; make install.
14. Nix expressions
all-packages.nix
{system ? builtins.currentSystem}:
rec {
stdenv = ... { inherit system; };
fetchurl = ...;
pkgconfig = ...;
dbus_glib = ...;
libxml2 = ...;
libxslt = ...;
getopt = ...;
nix = callPackage ../pkgs/tools/package-management/nix { };
dysnomia = import ../pkgs/tools/package-management/dysnomia {
inherit stdenv fetchurl getopt;
};
disnix = import ../pkgs/tools/package-management/disnix {
inherit stdenv fetchurl pkgconfig dbus_glib;
inherit libxml2 libxslt getopt nix dysnomia;
};
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Environments
Composes packages by calling them with the required
function arguments.
Function invocations are lazy – they are only evaluated
if needed.
Previous expression for Disnix that defines a function
is imported here.
All dependencies are composed in the same expression
as well.
15. Building Nix expressions
Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
16. Building Nix expressions
Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3
The Nix package manager builds disnix and all its
dependencies that have not been built yet.
Hash component is derived from all build inputs used to build
the package.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
20. Building Nix expressions
During a build of package many side effects are removed:
Most environment variables are initially cleared or set to
dummy values, such as PATH.
Environment variables, such as PATH, are configured to only
contain the specified dependencies.
Nix store paths prevent packages to be implicitly found in
many cases (unlike “traditional” systems using /usr/lib,
/usr/bin or C:WINDOWSSystem32).
Timestamps are set to 1 second after the epoch
Files in the Nix store are made read-only.
Optionally, builds can be performed in a chroot()
environment, improving purity
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
21. User environments
Users can have
different sets of
installed applications.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
22. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
23. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
24. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
25. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
PATH
/nix/.../profiles
current
42
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
26. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env --remove-generations old)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
27. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-collect-garbage)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
28. Hydra expression
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomia ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
version = builtins.readFile ./version;
src = dysnomia;
buildInputs = [ pkgs.getopt ];
};
build = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
version = builtins.readFile ./version;
src = tarball;
buildInputs = [ pkgs.getopt ];
});
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
29. Hydra expression
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomia ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
version = builtins.readFile ./version;
src = dysnomia;
buildInputs = [ pkgs.getopt ];
};
build = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
version = builtins.readFile ./version;
src = tarball;
buildInputs = [ pkgs.getopt ];
});
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Jobs
An Hydra expression is a function returing an attribute
set: rec{attr1 = value1; ...; attrn = valuen; }
Function parameters define variability points:
Locations of the Dysnomia, Nixpkgs collection Git
repositories
Target system architectures
Each attribute corresponds to a job.
Each value refers to a function performing a build or
test.
File is typically placed in the root folder of a source
package.
30. Building jobs from the command-line
Building a source tarball:
$ nix-build release.nix -A tarball
Building Dysnomia for 64-bit AMD Linux:
$ nix-build release.nix -A build.x86 64-linux
Building Dysnomia for Mac OS X (Nix delegates the build to
a Mac machine if the build is run on Linux and an external
machine is configured):
$ nix-build release.nix -A build.x86 64-darwin
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
31. Building jobs from Hydra
Create a project:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
32. Building jobs from Hydra
Create a jobset:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
33. Building jobs from Hydra
Create a jobset:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Jobs
All (but one) inputs are provided as function
arguments to the release expression.
One input is the package itself (dysnomia) that
contains the release.nix expression
34. Building jobs from Hydra
Evaluation results (job names correspond to those defined in
release.nix):
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
35. Hydra expression referring to other jobsets
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomiaJobset ?
import ../dysnomia/release.nix { inherit nixpkgs systems; }
, disnix ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = ...
build = pkgs.lib.genAttrs systems (system:
let dysnomia = builtins.getAttr system (dysnomiaJobset.build); in
with import nixpkgs { inherit system; };
releaseTools.nixBuild {
name = "disnix";
src = tarball;
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
};
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
36. Building jobs from Hydra
Use an input of type: ’Previous Hydra evaluation’:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
37. Hydra jobs I typically write
Source packages. Jobs that assemble tarballs, Zip files or
other archives containing the source code.
Binary packages. The actual builds for a variety of
architectures, such as i686-linux, x86 64-linux,
x86 64-darwin.
Program manuals. For example, building the manual from
Docbook.
Program documentation catalogs. Generating a
documentation catalog from the source code, e.g. using
javadoc, doxygen or JSDuck.
Unit tests. Running Unit tests, for example with JUnit or
mocha and producing a coverage report.
System integration tests. Composing NixOS Linux VMs with
all environmental dependencies, e.g. DBMS, web server etc,
and run tests inside them.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
38. Nix channel
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
39. Nix channel
Adding a channel:
$ nix-channel --add http://localhost/jobset/disnix/disnix-master/channel/latest
$ nix-channel --update
When running:
$ nix-env -i disnix
installing ‘disnix-0.3pre174e883b7b09da822494876d2f297736f33707a7’
these paths will be fetched (0.31 MiB download, 0.91 MiB unpacked):
/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3
fetching path ‘/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3’...
The build gets downloaded from the Hydra server, instead of being
built from source code.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
40. Nix package manager: Exercises
Check it out yourself!!!
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
41. Availability
Nix and Hydra are available as free and open source software under
the LGPLv2 and the GPLv3 licenses:
Nix: http://nixos.org/nix
Hydra: http://nixos.org/hydra
NixOS’ Hydra server: http://hydra.nixos.org
Nix can be used on any Linux distribution, NixOS, Mac OS X,
FreeBSD, and Windows (through Cygwin)
Hydra can be used on any Linux distribution
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
42. Related work
Using Nix while doing development:
Deploy development packages and composing an environment
in which they can be found
NixOS: http://nixos.org/nixos
Deploy an entire system configuration (Linux distribution) with
Nix.
System integration testing with NixOS
Efficiently compose networks of NixOS machines within a build
in which system integration tests can be performed
Disnix: http://nixos/disnix
(Re)deploy service-oriented systems into networks of machines
NixOps: http://nixos/nixops
Deploy networks of NixOS configurations to physical machines
or into the cloud
Automatically creates VM instances if needed
NiJS: https://www.npmjs.org/package/nijs
Compose Nix packages in JavaScript
Invoke JavaScript functions from Nix expressions
Very primitive stand-alone package manager
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People