Continuous Integration is like having a robot that cleans up after you: it installs your dependencies, builds your project, run your tests, and reports back to you. This presentation outlines two methods for CI: Travis and Jenkins.
6. Agenda
• What is Continuous Integration?
• Why use Continuous Integration?
• CI in Practice: Travis
• Coverage Thresholds with coverage.py
• CI in Practice: Jenkins
• Best Practices & Further Exploration
9. What is CI?
1. Checkout out the latest copy of your code
2. Install the dependencies
3. Run the tests
4. Report the outcome
Some server somewhere checks out your code, installs all your dependencies, runs your tests, and let’s
you know if it worked or not.
10. Testing in Django
# jmad/tunes/tests/test_views.py
from django.test import TestCase
class TunesViewTestCase(TestCase):
def test_index_view(self):
response = self.client.get('/')
self.assertEqual(response.status_code, 200)
self.assertTemplateUsed(response, 'index.html')
Here’s a quick primer on Django tests, for the uninitiated.
11. Testing in Django
# jmad/tunes/views/__init__.py
from django.views.generic import TemplateView
class IndexView(TemplateView):
template_name = "index.html"
# jmad/jmad/urls.py
from django.conf.urls import patterns, url
from tunes.views import IndexView
urlpatterns = patterns('',
url(r'^$', IndexView.as_view(), name='index'),
)
Here’s the code to make that test pass (assuming there’s an index.html in a template directory
somewhere).
12. Testing in Django
$ python manage.py test
.
---------------------------------------------------------------
-------
Ran 1 test in 0.029s
OK
Destroying test database for alias 'default'...
$
And here’s how you’d run the test suite for the project.
14. Why CI?
Drink more beer.
How can I drink more beer if the client calls me on Saturday night because the site is down?Bugs
should show up fast, and you should fix them before your code is deployed.
15. Why CI?
Your test suite is only useful if you run it.
You will forget to run your tests. Your collaborators will forget to run your tests. If you don’t run the
tests, you won’t know your code is busted when it’s time to deploy.
16. Why CI?
I run tests with SQLite, d@&$#t.
It’s fast. It’s there. If I want to build my project on my mom’s Mac Mini I don’t have to install
homebrew. Plus, running a test suite creates a new database everytime. Let the m1.small sitting in
Northern Virginia wait around for TestCase to create the PostgreSQL database. It doesn’t even like
beer.
17. Why CI?
Your project is not getting deployed to a Macbook.
Having said that, you need to run your tests in an environment like your production environment. Your
project may have wacky dependencies that you installed a long time ago on your Macbook.
19. Why CI?
# requirements.txt
...
PIL=1.1.7 # need to redeploy?
The time to learn about a neat new package replacing an old clunky one is NOT when you’re trying to
do an emergency redeployment.
23. http://www.scu.edu/profiles/?p=5184
More importantly, I know Bill Stevens. He’s jazz faculty at Santa Clara University. He wrote a book
called “Jazz Musicianship”. It’s a guide for jazz improvisation based on patterns that exists in many
different jazz tunes.
24. While we read headings like “Adding Callables to ModelAdmin Classes”...
26. Screenshot
JMAD is an archive of jazz music that’s been tagged with these different musical concepts. Basically
you search by concept, instrument, difficulty, etc., and you get back solos with those parameters
29. Travis: Setup
1. Sign in with your Github account
2. Select the repos you want Travis to work on
3. Add a .travis.yml file to the root of the repo
4. Commit
45. coverage.py
$ coverage run --source='.' manage.py test --settings=jmad.settings.base
Creating test database for alias 'default'...
...E
...
----------------------------------------------------------
Ran 4 tests in 3.353s
FAILED (errors=1)
Destroying test database for alias 'default'...
Run your tests with coverage...
46. coverage.py
$ coverage report --omit=*test*,*settings*
Name Stmts Miss Cover
-------------------------------------------------
jmad/__init__ 0 0 100%
jmad/urls 6 0 100%
jmad/wsgi 4 4 0%
manage 6 0 100%
people/__init__ 0 0 100%
people/admin 1 0 100%
people/models 1 0 100%
people/views/__init__ 4 0 100%
tunes/__init__ 0 0 100%
tunes/admin 1 0 100%
tunes/models 1 0 100%
tunes/templatetags/__init__ 0 0 100%
tunes/templatetags/add_css 5 0 100%
tunes/urls 3 0 100%
tunes/views/__init__ 5 0 100%
-------------------------------------------------
TOTAL 37 4 89%
... then generate a report to see what’s not being tested.
48. coverage.py
$ coverage report --omit=*test*,*settings*
--omit tells coverage to not report on a few things. Since we have multiple settings files, and we don’t
test our tests, we omit them both.
49. coverage.py
$ coverage report --omit=*test*,*settings* --fail-under=85
--fail-under take an integer, and compares that to the total percentage of coverage, and makes this
command return a 2 (anything but 0 is failure).
50. Travis and coverage.py
language: python
python:
- "3.3"
- "2.7"
# command to install dependencies
install:
- "pip install -r requirements/ci.txt"
# command to run tests
script:
- "coverage run source=’.’ manage.py test"
- "coverage report --omit=*settings*,*test*
--fail-under=85" # 85% coverage minimum
Swap out your script with these two lines: one to run the tests with coverage, and the other to run a
report that can fail. Now if we’re at 85% coverage and someone pushes new code without test
coverage, we’ll drop below 85% and get a failed build.
54. Installing Jenkins
$ wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war
$ java -jar jenkins.war
http://localhost:8080
Just get the .war file, run it, and hit port 8080 on your machine. You could deploy it behind Tomcat or
the like.
55. Or grab a VM
Installing Jenkins
I used a TurnKey Linux prebuilt VM.
56. Configuring Jenkins
You’ll need some plugins:
- Jenkins Violations Plugin
- Jenkins Git Plugin
- Jenkins Cobertura Plugin
Install a few plugins.
57. Configuring Jenkins
Got Selenium tests?
So, if you’re like me, you’ve got Jenkins on a headless version of Linux. We’ll have to do some magic to
get our Selenium tests to run.
58. Configuring Jenkins
# on the Jenkins box
$ sudo apt-get install iceweasel # install firefox
$ sudo apt-get install xvfb # frame buffer emulator
Install firefox (the package is called iceweasel for some reason).
59. Configuring Jenkins
# /etc/init.d/xvfb
#!/bin/bash
if [ -z "$1" ]; then
echo "`basename $0` {start|stop}"
exit
fi
case "$1" in
start)
/usr/bin/Xvfb :99 -ac -screen 0 1024x768x8 &
;;
stop)
killall Xvfb
;;
esac
Configure xvfb to run when the server starts
60. Configuring Jenkins
$ sudo chmod 755 /etc/init.d/xvfb
$ sudo shutdown -r now
make that file executable, and restart the server
61. Configuring Jenkins
$ pip install django-jenkins
INSTALLED_APPS = (
...
'django_jenkins',
)
...
JENKINS_TASKS = (
'django_jenkins.tasks.run_pylint',
'django_jenkins.tasks.with_coverage',
'django_jenkins.tasks.run_pep8',
# there are more of these
)
$ python manage.py jenkins # Jenkins will run this command
django-jenkins is a plugin that runs out tests and outputs the files Jenkins needs to show our build
stats. pip install and add some stuff to your settings.py
62. Configuring Jenkins
1. Configure a new test (name, description)
2. Give it your repo URL
3. Tell it how often to build
4. Tell it the commands to run
5. Configure where to save the reports
6. Click “Build Now”
Check out the tutorials in the “Resources” section of this slide deck for more on configuring your repo.
It’ll take about 15 minutes the first time.
63.
64.
65.
66.
67. Configuring Jenkins
#!/bin/bash
virtualenv -p python3.4 env
env/bin/pip install -r requirements/ci.txt
export SECRET_KEY='dnqsj22jdv9wjsldfub9'
export DISPLAY=:99
env/bin/python manage.py jenkins --settings=jmad.settings.ci
here’s what that command really should be
68.
69.
70. So what did we get?
Jenkins
I used a TurnKey Linux prebuilt VM.
71. Here’s a list of all the projects Jenkins knows to build
72. The project page for JMAD. Note the historic list of builds at the bottom left.
73. An individual build. That’s a list of commit messages under ‘changes’. And the link to the console
output.
77. Free as in beer and speech
Jenkins: Pros
Open. Extensible. Good for the spirit.
78. Plugins install like Wordpress
Jenkins: Pros
And by that I mean, it’s almost *too* easy. Just search for them from your installation and click
“install”.
79. Distributed builds
Jenkins: Pros
I haven’t done any work with this at all, but Jenkins supports a ‘master/slave’ mode, allowing a single
Jenkins instance to control many others. This would allow you to test a project on a bunch of different
platforms simultaneously. You can see how that would benefit the Django project itself, or other large
Python packages.
80. It’s your architecture.
Jenkins: Pros
Need to run Python compiled with some magical incantation? Need a special server utility installed?
Jenkins runs on an OS you control, so do what you gotta do.
81. It’s nobody else’s architecture.
Jenkins: Cons
That, of course, leads to our first con.
82. To paraphrase Uncle Ben, “With great power can come a whole lot of bullshit.”
83. 15 apt-get update
16 sudo apt-get install build-essential
17 apt-get install build-essential
18 apt-get install libsqlite3-dev
19 apt-get install sqlite3
20 apt-get install bzip2 libbz2-dev
21 ls
22 ls ..
23 mkdir src
24 cd src/
25 wget http://www.python.org/ftp/python/3.4.0/Python-3.4.0.tar.xz
26 tar xJf ./Python-3.4.0.tar.xz
27 cd Python-3.4.0/
28 ./configure --prefix=/opt/python3.4
29 make && make install
30 python3.4
31 ln -s /opt/python3.4/bin/python3.4 ~/bin/python3.4
32 ls ~
33 mkdir bin
34 ln -s /opt/python3.4/bin/python3.4 ~/bin/python3.4
35 ls
Jenkins: Cons
Here’s the first twenty lines of me fumbling through installing Python 3.4, pip, and virtualenv
84. 36 ls bin
37 rm -rf bin
38 mkdir ~/bin
39 ln -s /opt/python3.4/bin/python3.4 ~/bin/python3.4
40 python2.4
41 python3.4
42 virtualenv
43 ls /opt/python3.4/bin/
44 cd
45 ls
46 ls bin/
47 bin/python3.4
48 python3.4
49 ls /usr/bin/
50 ln -s /opt/python3.4/bin/python3.4 /usr/bin/python3.4
51 python3.4
52 rm -rf bin/
53 wget https://raw.github.com/pypa/pip/master/contrib/get-pip.py
54 python3.4 get-pip.py
55 apt-get install zlib
56 apt-get install zlib1g
Jenkins: Cons
... and here’s the next twenty lines...
85. 57 python3.4 get-pip.py
58 apt-get install zlib1g
59 apt-get install zlib-dev
60 apt-get install zlibc
61 python3.4 get-pip.py
62 apt-get install zlib1g-dev
63 python3.4 get-pip.py
64 apt-get install zlib-bin
65 python3.4 get-pip.
66 python3.4 get-pip.py
67 cd src/
68 ls
69 cd Python-3.4.0/
70 history
71 ./configure --prefix=/opt/python3.4
72 make && make install
73 apt-get install libssl-dev openssl
74 make && make install
75 which pip
76 cd ~
77 ln -s /opt/python3.4/bin/pip3.4 /usr/bin/pip3.4
78 pip3.4 install virtualenv
Jenkins: Cons
... and the next. So you’ll need some sysadmin chops.
86. And I still need to set up a mail server.
Jenkins: Cons
87. Git is a plugin
Jenkins: Cons
Again, it’s not hard to install plugins, but other tools are Git-centric, so it’s worth mentioning.
90. Best Practices
# jmad/settings/ci.py
INSTALLED_APPS = (
...
‘django_jenkins’
)
...
JENKINS_TASKS = (
‘django_jenkins.tasks.run_pylint’,
‘django_jenkins.tasks.with_coverage’,
‘django_jenkins.tasks.run_pep8’
)
Keep this stuff out of your production app (and your dev environment, for that matter).
93. Best Practices
#!/bin/bash
rm -rf env
virtualenv -p python2.7 env
env/bin/pip install -r requirements/ci.txt
export SECRET_KEY='dnqsj22jdv9wjsldfub9'
env/bin/python manage.py jenkins --settings=jmad.settings.ci
Toss the old env before you recreate it.
95. “Just-in-time Jenkins”
a.k.a.
“The AWS Miser”
a.k.a
“Big Testing”
Further Exploration
You could imagine a scenario in which, using one of the many DevOps tools available, you spin up and
AWS instance, install and configure Jenkins, set up your tests, report and then tear down the box. This
would be particularly useful if you wanted to test on a multidtude of platforms but didn’t want to pay
to keep them all up all the time.
96. Alternatives to Travis
• https://circleci.com/
• https://drone.io/
Further Exploration
Travis is not the only SaaS CI game in town. Drone.io works with BitBucket.
97. tox
Further Exploration
It’s a tool to test your project in multiple version of Python in one go, and can act as a front end to a CI
server. Has anyone here used tox?
109. Resources
Setting up Jenkins for Selenium tests
• http://www.labelmedia.co.uk/blog/setting-up-
selenium-server-on-a-headless-jenkins-ci-build-
machine.html
• http://www.installationpage.com/selenium/how-
to-run-selenium-headless-firefox-in-ubuntu/
• Just Google it