Testing Programmable Infrastructure with Ruby

@burythehammer
MATT LONG
TESTING PROGRAMMABLE
INFRASTRUCTURE (WITH RUBY)

InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations
/opencredo-ruby
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

@burythehammer
PROGRAMMABLE
INFRASTRUCTURE IS GREAT, BUT
WE'RE MISSING SOMETHING.
TESTING.

@burythehammer
I'M A TESTER
HELLO, I'M MATT
@burythehammer

@burythehammer
I WORK HERE ↑

@burythehammer
I AM NOT A
SYSADMIN

@burythehammer
WHAT IS
PROGRAMMABLE
INFRASTRUCTURE?

TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO THE
MANAGEMENT OF IT INFRASTRUCTURE
PROGRAMMABLE INFRASTRUCTURE IS..
THE INTERNET

WHAT DO WE MEAN BY THIS?
▸ Automated provisioning & conﬁguration
▸ Conﬁguration as code
▸ Version / source controlled

TOOLING EXAMPLES

@burythehammer
PROGRAMMABLE
INFRASTRUCTURE
IS AWESOME!
Credit: Vault Boy, Bethesda Softworks

@burythehammer
BUT IT CAN GET
COMPLEX

@burythehammer
TESTING IS USED TO
MITIGATE COMPLEXITY
& RISK

@burythehammer
BUT INFRA TESTING IS RARE
Credit: Gunshow, KC Green

@burythehammer
TESTING A
CLOUD BROKER
AN INFRASTRUCTURE HEAVY PRODUCT

WE WANT TO MOVE TO THE CLOUD...
BUT WE'RE WARY OF LOCK IN
Large organisation

USERS
CLOUD BROKER

BENEFITS
▸ Quick, easy provisioning
▸ one team previously took 3 months
▸ Don't need to work at a low level
▸ Templates for common dev environments
▸ Built in best practice: monitoring, security
▸ Track spending

@burythehammer
THIS IS A REALLY
COMPLICATED
APPLICATION

WORKFLOW
▸ Log into Web UI
▸ Fill in information about environment
▸ Broker creates and bootstraps resources
▸ SSH into resources

WEB TESTING
▸ Log into Web UI
▸ Fill in information about environment

???

INFRASTRUCTURE TESTING

@burythehammer
HOW DO YOU TEST
INFRASTRUCTURE?

WHAT TO TEST?
Deployment
scripts unit tests?
Does the VPN server work?
Can instances  
access one another?
Are services running?
Linting?

@burythehammer
THIS SEEMS
FAMILIAR..

Does the VPN box work? 
Can I SSH into a server?
Do our deployment scripts work? 
Linting?
Are services running?
ANOTHER TESTING PYRAMID?
credit: Ubuntu dev quality guide 
https://developer.ubuntu.com/en/phone/platform/quality/
Can instances access one another?
expensive, slow
cheap, fast

@burythehammer
UNIT TESTING
▸ Bash scripts
▸ Ansible scripts
▸ Terraform scripts

@burythehammer
LINTING
▸ Quick sanity check
▸ Available in all tools
▸ Run in CI before committing

@burythehammer
UNIT TESTING IS HARD
▸ Cultural issues
▸ Technical issues
▸ Return on investment

@burythehammer
INTEGRATION
TESTING
▸ Packages installed
▸ Services running
▸ Ports listening

▸ Ruby / RSpec based
▸ Great community
▸ Very readable
▸ Very quick!
▸ Can SSH into instances
SERVERSPEC

SERVERSPEC EXAMPLE
describe package('jenkins') do
it { should be_installed }
end
describe service('jenkins') do
it { should be_enabled }
it { should be_running }
end
describe port(8080) do
it { should be_listening }
end

@burythehammer
ACCEPTANCE
TESTING
▸ SSHing into machines
▸ Using applications
▸ e.g. samba, openvpn

@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
▸ Executable speciﬁcations
▸ Both test & business can read it
▸ Available in Ruby, Java, C#... etc
▸ Great reporting
▸ Good CI integration
CUCUMBER

 
Scenario: Searching for Wikipedia
Given I am on the website "www.google.com"
When I search for "Wikipedia"
Then the first link should be "www.wikipedia.org"
CUCUMBER EXAMPLE

RUBY
▸ Fantastic testing community
▸ More suitable for SSHing into boxes
▸ "Win RM" gem
▸ Ops already familiar with it
▸ Reduces tech stack

TOOLS WE DIDN'T USE
▸ Bats
▸ ShUnit2
▸ Goss
▸ Inspec / TestInfra
▸ AWS Spec
▸ Test Kitchen

USERS
WEB TEST FRAMEWORK

USERS
WEB TESTS
https://github.com/opencredo/test-automation-quickstart

USERS
INFRASTRUCTURE TEST FRAMEWORK

INFRASTRUCTURE TESTS
Serverspec

INFRASTRUCTURE TESTING STACK
/ Serverspec
Linting tools

SERVERSPEC SMOKE TESTS
▸ Run before everything else
▸ Really quick
▸ Catches obvious errors
▸ Not complex tasks

Background: 
Given environment has been created 
And the following user details: 
| user_alias | username | public_key | 
| userA | testuser | test |
 
 
Scenario: IPA - Login via SSH Key authentication succeeds 
Given user "userA" is authorised to access environment vms 
When user "userA" starts ssh session in host "env"
 
Then I should be able to echo "hello world" 
CUCUMBER FOR ACCEPTANCE TESTING

Background: 
 
 
 
 
Cloud broker APIs

Background: 
 
 
 
 
Standard Ruby

Background: 
 
 
 
 
RSpec assertions

UNDER THE CUCUMBER, PLAIN RUBY
Then(/^I should be able to echo "([^"]*)"$/) do |text|
cmd = "echo #{text}"
output = @session.exec!(cmd)
close_ssh(@session)
expect(output.to_s.strip).to eql(text)
end

@burythehammer
LESSONS
LEARNED
THE GOOD, THE BAD, AND
THE UGLY

THE GOOD
▸ Specialised tests for each layer
▸ Quick, expressive ServerSpec
tests
▸ Power of a full programming
language for user tests
▸ ... it's also totally doable!
Credit: Overwatch, Blizzard Entertainment

THE BAD
▸ Over reliance on
acceptance tests
▸ Context switching with
two suites
▸ Out of my comfort zone
Credit: Futurama, Fox

THE UGLY
▸ Starting infrastructure is SLOW.
▸ It's expensive...

@burythehammer
IT WAS WORTH IT
DESPITE ALL THAT

@burythehammer
TESTING IS
IMPORTANT
BUT OFTEN IGNORED

@burythehammer
TESTERS AND OPS
SHOULD WORK TOGETHER
WE NEED TO GET OUT OF
OUR COMFORT ZONES

@burythehammer
TOOLS EXIST
BUT BE PREPARED
TO HACK

THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO
MANAGEMENT OF IT INFRASTRUCTURE
PROGRAMMABLE INFRASTRUCTURE IS..

@burythehammer
TESTING IS A SOFTWARE
DEVELOPMENT METHOD
WE SHOULD APPLY IT TO
INFRASTRUCTURE

@burythehammer
THANKS!
QUESTIONS?
matt.long@opencredo.com
<-- sponsors booth on 5th ﬂoor!
@burythehammer

Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
opencredo-ruby

Testing Programmable Infrastructure with Ruby

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Testing Programmable Infrastructure with Ruby

Similaire à Testing Programmable Infrastructure with Ruby (20)

Plus de C4Media

Plus de C4Media (20)

Dernier

Dernier (20)

Testing Programmable Infrastructure with Ruby