Comparing different Testing styles in Scala.

1. Testing with Style -
Andreas Neumann
Senior Software Engineer Search

2. ‹Nr.›
CONTENTS / ASSERTIONS

3. ‹Nr.›
Aims
▪ Tests are code that test other code
▪ Tests prove or falsify certain assumptions
▪ general structure for tests
▪ Readability is important => doubles as documentation
▪ Examples two main Scala Testing Frameworks: Specs2 ,
ScalaTest

4. ‹Nr.›
Contents
▪ Structuring your project : A project Blueprint
▪ Run tests : I have written the code, what now ?
▪ Testing styles : So many to chose from
▪ Test Results : Get and interprete results

5. ‹Nr.›
PROJECT STRUCTURE

6. ‹Nr.›
Project Structure
▪ Scala/JavaProject:
▪ /src/test
▪ /src/it
▪ Subfolders:
▪ scala : Tests written in Scala
▪ java : Tests written in Java
▪ resources: Files needed for
testing
▪ these folders are not reachable out of main but can use
everything in main
▪ will normally not be included in Artifacts

7. ‹Nr.›
Structuring Tests
▪ tests can be structured in packages
▪ tests should mimic the main package
▪ to allow for portability there should be 1 .. n test files for 1
implementation file

8. ‹Nr.›
RUNNING YOUR TESTS

9. ‹Nr.›
Running Tests
▪ Tests can be run in many different ways
▪ we will look at
▪ CI
▪ IDE
▪ sbt

10. ‹Nr.›
CI
▪ CI does that for you
▪ runs complete set of tests
▪ run by git push
▪ run by hand
▪ for all Devs

11. ‹Nr.›
IDE
▪ built in support in every major IDE
▪ often through JUnit Integration
▪ can run single files or suites
▪ tad slow, no looping

12. ‹Nr.›
SBT
▪ Run all tests : test
▪ Run all tests in a specific project: <PROJECT>/test
▪ Run only one specific test: testOnly
▪ Run only tests failed before: test-quick
▪ Looping: ~
activator testing_styles/test
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
[info] Loading global plugins from /Users/anneumann/.sbt/0.13/plugins
[info] Loading project definition from /Users/anneumann/test-wars/project
[info] Set current project to test-wars (in build file:/Users/anneumann/test-wars/)
[info] PersonFeatureSpec:
[info] As a Developer
[info] I want to have a Person
[info] which can be part of the Test Wars Universe
[info][info] Total for specification PersonSpecMutable
[info] Finished in 56 ms
[info] 7 examples, 0 failure, 0 error
[info]
[info] ScalaTest
[info] Run completed in 1 second, 404 milliseconds.
[info] Total number of tests run: 14
[info] Suites: completed 3, aborted 0
[info] Tests: succeeded 14, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 36, Failed 0, Errors 0, Passed 36, Pending 3
[success] Total time: 2 s, completed Apr 15, 2015 1:03:14 PM

13. ‹Nr.›
IDE with SBT and CI

14. ‹Nr.›
DIFFERENT WAYS OF WRITING
YOUR TESTS - BY EXAMPLES

15. ‹Nr.›
What we want to test
Person
can be created without exception
Person default values
is Neutral by default
is no Jedi by default
has default aiming prob of 50%
Person can change sides ad libitum
to the empire
to Rebels

16. ‹Nr.›
Classic : JUnit like
▪ Looks like normal code
▪ Workhorses
▪ Scalatest : FunSuite
▪ test(„description") { …
}
▪ assert( cond => Boolean
)
▪ pro: nothing new to learn,
just functions
▪ con: looks like code, can
get messy
class PersonTestClassic extends FunSuite {
test("Person can be created without exception") {
assert( Person("Nobody").isInstanceOf[Person] == true )
}
test("Person is Neutral by default") {
assert( person.side == Neutral )
}
test("Person is no Jedi by default") {
assert( person.isJedi == false )
}
test("Person has default aiming prob of 50%") {
assert( person.aim == Probability(0.5) )
}
test("Person can change sides ad libitum to the Empire") {
val anakin = Person("Anakin", isJedi = true)
anakin.side = Empire
assert( anakin.side == Empire )
}
test("Person can change sides ad libitum to the Rebels") {
val han = Person("Han Solo")
assert( han.side == Neutral )
han.side = Rebels
assert( han.side == Rebels )
}
def person = Person("Honk")
}

17. ‹Nr.›
QA loves this - FeatureSpec
▪ Focus on description
▪ Workhorses
▪ FeatureSpec
▪ Given - When - Then
▪ pro: Focus on Functionality /
Requirements
▪ con:,mixes description and code,
hard to debug, ordering
class PersonFeatureSpec extends FeatureSpec with GivenWhenThen {
info("As a Developer")
info("I want to have a Person")
info("which can be part of the Test Wars Universe")
info("which has defaults and can change sides")
feature("Person") {
scenario("Creating a Person gives Person with defaults") {
Given("a Person created with just the name")
val person = new Person("Honk")
Then("the person should have defaults")
assert(person.isJedi == false)
assert(person.aim == Probability(0.5))
}
}
feature("Keep it interesting") {
scenario("A person can change sides") {
Given("Han Solo")
val han = new Person("Han Solo")
assert(han.side == Neutral)
When("han sees the good in the
Rebellion and befriends Luke he changes sides")
han.side = Rebels
Then("Han is part of the Rebellion")
assert(han.side == Rebels)
}
}
}

18. ‹Nr.›
A little DSL - FlatSpec with Matchers
▪ More like natural language
▪ Workhorses
▪ Scalatest FlatSpec
▪ ShouldMatchers : DSL
▪ pro: more natural, can be
understood by non
programmers,not boolean
centric
▪ con: What is tested
hidden by code
class PersonFlatTest extends FlatSpec with ShouldMatchers {
"A Person" should "be created without exception" in {
Person("Nobody") shouldBe a [Person]
}
it should "be Neutral by default" in {
person.side should be(Neutral)
}
it should "be no Jedi by default" in {
person.isJedi should be(false)
}
it should ("have default aiming prob of 50%") in {
person.aim should be( Probability(0.5) )
}
"Person can change sides ad libitum " should "to the Empire" in {
val anakin = Person("Anakin", isJedi = true)
anakin.side = Empire
anakin.side should be( Empire )
}
it should "to the Rebels" in {
val han = Person("Han Solo")
han.side should be( Neutral )
han.side = Rebels
han.side should be ( Rebels )
}
def person = Person("Honk")
}

19. ‹Nr.›
Another little DSL: Specs2 mutable
▪ like natural language
▪ Workhorses
▪ mutable.Spec
▪ specs2 DSL
▪ pro: natural, can be
understood by non
programmers, not
boolean centric
▪ con: code can hide
tests
class PersonSpecMutable extends Specification {
"A Person" should {
"be created without exception" in {
Person("Nobody") must not throwA(new Exception)
}
}
"Person default values" should {
"be Neutral by default" in new TestPerson {
side mustEqual Neutral
}
"be no Jedi by default" in new TestPerson {
isJedi must beFalse
}
"have aiming prob of 50% " in new TestPerson {
aim mustEqual Probability(0.5)
}
"have default chance of evading 50%" in new TestPerson {
evade mustEqual Probability(0.5)
}
}
"Person can change sides ad libitum" should {
"to the empire" in {
val anakin = Person("Anakin")
anakin.side mustEqual Neutral
anakin.side = Empire
anakin.side mustEqual Empire
}
"to Rebels" in {
val han = Person("Han Solo")
han.side mustEqual Neutral
han.side = Rebels
han.side mustEqual Rebels
}
}
class TestPerson(name: String = "Honk") extends Person(name) with Scope
}

20. ‹Nr.›
Functional Acceptance Style
▪ personal favorite
▪ Worhorses:
▪ Specs2 Specification
(immutable)
▪ String interpolation
▪ DSL with Matchers
▪ pro: clear distinction
requirements/description
code, functional, readable
by no coders ( upper Part )
▪ cons: Learning curve
class PersonSpec extends Specification {def is = s2"""
Person
can be created without exception $create
Person default values
is Neutral by default $defaultSide
is no Jedi by default $isTheForceWithHim
has default aiming prob of 50% $defaultAim
has default chance of evading 50% $defaultEvade
Person can change sides ad libitum
to the empire $becomeEvil
to Rebels $becomeRebel
"""
def create = Person("Nobody") must not throwA(new Exception)
def defaultSide = person.side mustEqual Neutral
def isTheForceWithHim = person.isJedi must beFalse
def defaultAim = person.aim mustEqual Probability(0.5)
def defaultEvade = person.evade mustEqual Probability(0.5)
def becomeEvil = {
val anakin = Person("Anakin")
anakin.side.mustEqual(Neutral).and {
anakin.side = Empire
anakin.side mustEqual Empire
}
}
def becomeRebel = {
val anakin = Person("Han Solo")
anakin.side.mustEqual(Neutral).and {
anakin.side = Rebels
anakin.side mustEqual Rebels
}
}
def person = Person("Honk")
}

21. ‹Nr.›
TEST RESULTS

22. ‹Nr.›
Test-Results
▪ Test Results need to be readable by humans
▪ Tests Results need to be machine readable !
▪ we will look at
▪ Terminal
▪ HTML
▪ JUnit-XML

23. ‹Nr.›
Test Results: Terminal
▪ for humans
▪ some color support
▪ green, yellow, blue, red
▪ result at the end
> testOnly SpaceShipSpec
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for test-wars/test:testOnly
[info] ScalaTest
[info] Run completed in 12 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for universe/test:testOnly
[info] SpaceShipSpec
[info] A spaceship
[info] + has shield
[info] + has attack power
[info] + has a aim, which defaults to 50% accuracy
[info] + has a chance to evade, which defaults to 50%
[info] + belongs to a side which by default is Neutral
[info]
[info] Spaceship Shield
[info] + a ship with shield left is ok
[info] + a ship with shield below 0 is broken
[info] + ship armor can be changed which affects the isOK state
[info]
[info] Spaceship Battle
[info] * a ship can engage another ship will not end in an endless loop PENDING
[info] * it will not engage if it is not Ok PENDING
[info] * after being engaged by another ship it will engage the other ship
[info] it will engage the other ship till one ship is no longer ok PENDING
[info]
[info] Total for specification SpaceShipSpec
[info] Finished in 46 ms
[info] 11 examples, 0 failure, 0 error, 3 pending
[info]

24. ‹Nr.›
Test Results: HTML
▪ depends on Testing Framework
▪ may need A LOT OF project configuration
▪ can also be used to create documentation
▪ console-output needs to be readded
//HTMLOutput
(testOptions in Test) ++= Seq(
Tests.Argument(TestFrameworks.Specs2, "html"),
Tests.Argument(TestFrameworks.ScalaTest, "-h",
"target/scalatest/html")
)

25. ‹Nr.›
Tests Results: Machine Readable
▪ machine readable
▪ used by many CIs
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite
errors="0" failures="0" hostname="mb0141417118.local" name="person.PersonFlatTest" tests="6" time="0.059"
timestamp="2015-04-15T13:19:05">
<properties>
<property name="jline.esc.timeout" value="0"> </property>
<property name="java.runtime.name" value="Java(TM) SE Runtime Environment">
</property>
<property
name="sun.boot.library.path" value="/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib">
</property>
<property name="java.vm.version" value="25.25-b02"> </property>
<property name="user.country.format" value="DE"> </property>
<property name="gopherProxySet" value="false"> </property>
<property name="java.vm.vendor" value="Oracle Corporation"> </property>
<property name="java.vendor.url" value="http://java.oracle.com/">
</property>
<property name="path.separator" value=":"> </property>
<property name="java.vm.name" value="Java HotSpot(TM) 64-Bit Server VM">
</property>
<property name="file.encoding.pkg" value="sun.io"> </property>
<property name="user.country" value="US"> </property>
<property name="sun.java.launcher" value="SUN_STANDARD"> </property>

26. ‹Nr.›
Thank YOU for YOUR participation
code can be found at:
https://github.com/daandi/test-wars/

27. ‹Nr.›
More to come, let me know what you want to hear about:
▪ (Mocking, Stubbing and Stabbing)
▪ Tests composability and inheritence
▪ Test Factories
▪ How do I test xy ( JSON, Futures, Web)
▪ Stubs, Mocks and Fixtures *
▪ Integration
▪ Polyglot testing
▪ Write testable code *
▪ Property based Testing (you won’t get away :)

28. ‹Nr.›
TRIVIA - How is that connected to the talk ?