presented at NorDev conference 2016 (Norwich) and also at Agile on the Beach conference 2016 (Falmouth)

1. twitter : @JonJagger
email : jon@jaggersoft.com
blog : http://jonjagger.blogspot.co.uk/
charity : http://cyber-dojo.org
the design and
evolution of
cyber-dojo
Hi. I'm Jon Jagger, a self-employed software consultant. I love fly fishing for salmon fishing. If you hire me and you're near a salmon river I'll reduce my rate!

2. late 2009(about tea-time)
Late in 2009 I was doing some consultancy in Oslo.

3. One evening I attended a code dojo being held in the Scotsman pub.

4. In case you're not familiar with the word, Dojo is the Japanese name for the place where martial artists meet to practice their martial art. So a code dojo is where coders
meet to practice coding.
Uncle Bob was the guest of honour. He selected the battleships coding exercise and also specified the language (I think it was Ruby) and the test framework. There were
about 30 developers - split into 6 teams of 5 people. We then had about 90 mins to code, after which each team would connect their laptop to a projector and show their
work. At least, that was the plan.

5. Somewhat predictably, most of the groups struggled to connect their laptop to the projector. Quite a lot of time was wasted.

6. Two groups didn't have much to show as they had spent more than half the time struggling to install the language and test-framework Uncle Bob specified. But
eventually everyone had shown their work and Uncle Bob pronounced one group the winners. Which was a bit strange.

7. In my hotel room that night I was thinking about the wasted time. Suppose everyone had been working in a shared cloud-based environment? There would have been
nothing to install. Only Bob's laptop would need connecting to the projector. The more I thought about this idea the more I liked it.

8. At some points in your life there are times where you decide something is worth spending your time on. This was a moment for me. I decided to build an IDE, an
Interactive Dojo Environment.
I rolled my sleeves up and started learning lots of new stuff I'd need. I didn't know any Ruby, HTML, CSS, Rails, Javascript, etc.

9. https://github.com/cyber-dojo/web
It's called it cyber-dojo.
It's open-sourced on github.
The first commit was in January 2010.
It's now 2016 and 8888 commits later.
Things don't come into existence instantly!

10. DEMO
Me demoing USING cyber-dojo.

11. Cyber-dojo is completely free for non-commercial use.
For commercial use you need to buy a license.

12. http://blog.cyber-dojo.org/2015/08/cyber-dojo-foundation.html
All license fees go to a registered charity (cyber-dojo Foundation) which buys Raspberry Pi computers for kids.

13. 40,000+ practice-sessions
Of course, even if you're non-commercial, your welcome to make a donation too :-)

14. VALUE
SYSTEM?
That's a bit about its evolution. Now a bit about its design...

15. Interactive
Dojo
Environment
Integrated
Development
Environment
• make money
• individual based
• finish focused
• test driven?
• work faster
• time pressure
• don't repeat
• have fun
• team based
• learning focused
• test driven
• work slower
• no time pressure
• repeat
It was clear to me from the beginning that a cloud-based environment designed for coding dojos should be different to a regular development environment....

16. • team based • individual based
• no login
• anonymity
No login. No username, password. Anonymity is very important. When reviewing we cannot talk about Bert and Ernie's code. We can only talk about the Lion's code.
Bert and Ernie can decide whether or not they want to reveal that they are the Lion.

17. • learning focused • finish focused
I have come to realise that developers derive enormous value from simply having their steps replayed to them. For example this is a real example from some consulting I
did recently in Bangalore. From the burst of green traffic lights it looks like the koala is refactoring. They are refactoring the duplication out of their tests. Nice. Although,
as another developer pointed out, the use of the word if in the assert helper is a bit dubious.

18. tests ran
one or more failed
tests ran
all passed
tests did not run
I extend the standard red/green fail/pass colour scheme.
I complete the traffic-light by using amber for cases when the tests do not run (eg syntax error)

19. {
"visible_filenames": [
"README",
"Hiker.feature",
"Hiker.java",
"HikerStepDef.java",
"RunCukesTest.java",
"cucumber.xml",
"cyber-dojo.sh"
],
"filename_extension" : ".java",
"display_name": "Java, Cucumber-Spring",
"image_name": "cyberdojofoundation/java_cucumber_spring",
"red_amber_green" : [
"lambda { |output|",
" return :red if /FAILURES!!!/.match(output)",
" return :green if /OK ((d*) tests)/.match(output)",
" return :amber",
"}"
]
}
.../Java/Cucumber-Sprint/manifest.json
Determining the red/amber/green static happens in a simple function held in the language+test_framework's manifest. It's typically implemented using simple regular
expression matchers.

20. One of the things cyber-dojo is very effective at (which I had not anticipated) is how well a short 45 minute practice session can mimic problems normally associated with
large projects typically running for months and years. For example, waterfall development, when stuff is endlessly 90% done and overruns.
In the above dashboard (from the same trip to Bangalore) note that the two animals with the fewest traffic lights are the frog and the snake. Note the time-gap (left to
right) before their first visible traffic-light, which, almost inevitably, is amber. Note too the burst of ambers following this initial amber as they try to "recover". The frogs
just got to green in time (lucky) but the snakes did not (very typical). Again, inevitably, the snakes ask for a little bit more time! They're almost done, they say. But no.
There is no more time. We are not shipping anything!

21. Reminder to drink so as not to lose my voice.

22. How does (and did) it work under the hood?

23. 2010 - No security!
(frequent backups)
In cyber-dojo arbitrary code is written in the browser and run on the server!!!! This has obvious security implications.
Yet the first version had no security. Really!
It was the simplest thing that could possibly work.
I had mostly built cyber-dojo for myself.
When I used cyber-dojo during paid consultancy I would simply spin up my own personal server.

24. Initially it was just a bare server with rails installed.

25. .py
Python
pytest
apt-get update && apt-get install --yes
libsqlite3-dev
sqlite3
bzip2
libbz2-dev
wget
wget http://www.python.org/ftp/python/3.3.5/
Python-3.3.5.tar.xz
tar xJf ./Python-3.3.5.tar.xz
cd ./Python-3.3.5 && ./configure --prefix=/opt/
python3.3
cd ./Python-3.3.5 && make
cd ./Python-3.3.5 && make install
ln -s /opt/python3.3/bin/python3.3 /bin/python
apt-get update && apt-get install -y
python-pip
pip install -U pytest
Python
pytest
It really was just the server. For example, to support Python using pytest (on the browser), then Python and pytest had to be installed on the server.

26. .py
.py
Python
pytest
Python
pytest
When the dev hit the test button on the browser the source files were copied on to the hard-disk of the server and run directly on the server! There was no security. No
chroot. Nothing!

27. $ cd /
$ rm -rf *
2013
This worked fine for several years on the public cyber-dojo.org server until one day someone ran the above code! They sent me a nice email saying how sorry they were,
that they had just assumed it wouldn't work!!
It was an ex server.

28. 0.3
I looked for ways to provide security. A new, little known (at that time) technology called Docker caught my eye. It was at about 0.3 then. Docker went open source in
March of 2013.

29. FROM cyberdojofoundation/build-essential
MAINTAINER Jon Jagger <jon@jaggersoft.com>
RUN apt-get update && apt-get install --yes
libsqlite3-dev
sqlite3
bzip2
libbz2-dev
wget
RUN wget http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tar.xz
RUN tar xJf ./Python-3.3.5.tar.xz
RUN cd ./Python-3.3.5 && ./configure --prefix=/opt/python3.3
RUN cd ./Python-3.3.5 && make
RUN cd ./Python-3.3.5 && make install
RUN ln -s /opt/python3.3/bin/python3.3 /bin/python
RUN apt-get update && apt-get install -y
python-pip
RUN pip install -U pytest
Dockerfile
Now the commands to install Python and pytest go into a file called Dockerfile.

30. cyberdojo/python_pytest
$ docker build -t cyberdojo/python_pytest .
FROM cyberdojofoundation/build-
essential
MAINTAINER Jon Jagger <jon@jaggersoft.com>
RUN apt-get update && apt-get install --yes
libsqlite3-dev
sqlite3
bzip2
libbz2-dev
wget
RUN wget http://www.python.org/ftp/python/
3.3.5/Python-3.3.5.tar.xz
RUN tar xJf ./Python-3.3.5.tar.xz
RUN cd ./Python-3.3.5 && ./configure --
prefix=/opt/python3.3
RUN cd ./Python-3.3.5 && make
RUN cd ./Python-3.3.5 && make install
RUN ln -s /opt/python3.3/bin/python3.3 /bin/
python
RUN apt-get update && apt-get install -y
python-pip
RUN pip install -U pytest
Dockerfile
Then, on a system with docker installed, you build your named docker image from the Dockerfile.
You've then got your image.

31. cyberdojo/python_pytest
.py .py
Python
pytest
Every time the dev presses the test button in the browser, the rails server spins up a docker-container from the appropriate docker-image (in this case the one for python
+pytest), copies the files into the container, and runs the test inside the container, grabs the output, and then destroys the container (but not the image). Creating a new
container from an image is almost instantaneous!!

32. The images for the 50+ language&test_framework combinations offered on the setup page are all stored on the docker hub and can be pulled freely by anyone who has
installed docker.

33. more animals!
2015
At first there were only 16 animals. Several people asked to be able to run practice sessions with a hundred or so participants and 16 was not enough. I needed more
animal images! My friend Dymtro Mindra paid a professional artist (Nadya Sivers) to draw 64 new images. Aren't they great :-)

34. setting up
your own
server was
lengthy and
error-prone
The biggest remaining problem was for people who wanted to run their own servers. Installing the rails server was lengthy and error prone.

35. cyberdojo/python_pytest
.py cyberdojo/web
.py
Python
pytest
So I put the rails server into a docker image as well!
(While I was at I put an nginx front end on too.)

36. cyberdojo/python_pytest
.py cyberdojo/web:1.12.1
.py
Python
pytest
1.12.1
Eagle eyed readers will spot there is a problem putting the rails server inside a docker image. The rails server issues docker commands to run the tests submitted from
the browser. But docker is installed on the server, not inside the image holding the rails server!
So I also had to install docker inside the web image. This is sometimes called docker-in-docker. This creates a version dependency. The version of docker installed on the
server (which spins up the web container) must match the version of docker installed inside the web container (which spins up the python_pytest container).
Now installation is mostly installing docker and pulling the pre-built cyber-dojo docker images.

37. FROM cyberdojo/user-base
MAINTAINER Jon Jagger <jon@jaggersoft.com>
...
USER root
RUN apk --update add
ruby ruby-irb ruby-io-console ruby-bigdecimal tzdata
bash
ARG CYBER_DOJO_HOME
USER root
RUN mkdir -p ${CYBER_DOJO_HOME}
COPY Gemfile ${CYBER_DOJO_HOME}
RUN apk --update
add --virtual build-dependencies
build-base
ruby-dev
openssl-dev
postgresql-dev
libc-dev
linux-headers
&& gem install bundler --no-ri --no-rdoc
&& cd ${CYBER_DOJO_HOME} ; bundle install --without development test
&& apk del build-dependencies
...
Dockerfile
cyberdojo/web:1.12.1
docker
build
Here's part of the Dockerfile for the cyber-dojo web image. It shows the commands to install ruby and setup the rails server.

38. FROM cyberdojo/user-base
MAINTAINER Jon Jagger <jon@jaggersoft.com>
...
ARG DOCKER_VERSION
USER root
RUN apk --update add curl
&& curl -OL https://get.docker.com/builds/Linux/x86_64/
docker-${DOCKER_VERSION}.tgz
&& tar -xvzf docker-${DOCKER_VERSION}.tgz
&& mv docker/* /usr/bin/
&& rmdir /docker
&& rm /docker-${DOCKER_VERSION}.tgz
&& apk del curl
...
Dockerfile
cyberdojo/web:1.12.1
docker
build
Here's part of the Dockerfile for the cyber-dojo web image. It shows the commands to install docker!

39. A bit about testing.

40. #!/bin/sh
set -e
my_dir="$( cd "$( dirname "${0}" )" && pwd )"
cd ${my_dir}/../docker/web-base
./build-image.sh
cd ${my_dir}/../docker/web
./build-image.sh
cd ${my_dir}/../cli
./cyber-dojo down
./cyber-dojo start-point ls --quiet | grep 'languages' && ./cyber-dojo start-point rm languages
./cyber-dojo start-point ls --quiet | grep 'exercises' && ./cyber-dojo start-point rm exercises
./cyber-dojo start-point ls --quiet | grep 'custom' && ./cyber-dojo start-point rm custom
./cyber-dojo start-point ls
./cyber-dojo start-point create languages --dir=./../../start-points-languages
./cyber-dojo start-point create exercises --dir=./../../start-points-exercises
./cyber-dojo start-point create custom --dir=./../../start-points-custom
./cyber-dojo start-point ls
./cyber-dojo up
cid=`docker ps --all --quiet --filter "name=cyber-dojo-web"`
docker exec ${cid} sh -c "cd test && ./run.sh"
done=$?
exit $done
test/build_and_run.sh
I have rails installed locally on my macbook. However, I do not run the tests locally. That would be simple, but it would be a trap. This is because the code being tested
would not be running inside a docker container. Yet when the code really runs (after being deployed) it always runs inside a docker container. So instead I run a script
which rebuilds the docker web image, starts a new web container, shells into this container, and runs the tests.

41. ...
env:
global:
- DOCKER_ENGINE_VERSION=1.12.1
- DOCKER_COMPOSE_VERSION=1.8.0
- DOCKER_DIR=${TRAVIS_BUILD_DIR}/docker
before_install:
# list docker-engine versions
- apt-cache madison docker-engine
# install specific version of docker-engine
# install specific version of docker-compose
...
.travis.yml
github has an excellent feature whereby every commit can trigger a Travis continuous integration (CI) pipeline. I use this on the cyber-dojo web repo. The script github
runs installs specific versions of docker and docker-related tools...

42. ...
script:
# Build the cyber-dojo web image
- cd ${DOCKER_DIR}/web
- ./build-image.sh ${DOCKER_ENGINE_VERSION}
- cd ${TRAVIS_BUILD_DIR}/cli
- ./cyber-dojo up
- cid=`docker ps --all --quiet --filter "name=cyber-dojo-web"`
- docker logs cyber-dojo-web
- docker inspect -f {{.State.Running}} ${cid}
- docker ps -a
- docker pull cyberdojofoundation/gcc_assert:latest
- docker pull cyberdojofoundation/ruby_test_unit:latest
.travis.yml
- docker exec ${cid} sh -c "cd test && ./run.sh"
...and then builds the web image, starts a new web container, shells into this container, and runs the tests. Just like when I run the tests myself :-)

43. Here's an example of the output you get from travis CI when the tests are run inside the docker container.

44. This is the dashboard for the Travis Continuous Integration. The various github repos are listed on the left. On the right are results from recent commits on the selected
github repo (web).

45. thank yous
Byran Wills-Heath Nadya Sivers Seb Rose
Olve Maudal Dymtro Mindra Mike Long
A big thank you also to these people who have all helped with cyber-dojo in various ways.

46. twitter : @JonJagger
email : jon@jaggersoft.com
blog : http://jonjagger.blogspot.co.uk/
charity : http://cyber-dojo.org
that's all
folks
That's it. Thanks for listening. I hope I've sowed a few seeds. Have a great conference.