1. import org.scalacheck._
BeScala – February 2012
Gilles Scouvart

2. Once upon a time…
• Three scientists (an astrophysicist, a physicist and a
mathematician) are in a bus to a conference in
Scotland
• Shortly after the border, they see a black sheep on a
hillside.
• The astrophysicist declares: « In Scotland, all sheep are
black! »
• The physicist then says: « Hm, all we can say is that in
Scotland, some sheep are black. »
• The mathematician sighs and concludes: « All we can
tell is that in Scotland, there exists one sheep with at
least one black side! »

3. The moral of the story
• We’re often relying on a limited set of simple
examples to validate our assumptions
• But our implementation might totally overfit
our test set (particularly in TDD)
• And edge cases might go completely
unnoticed…
Can we do better?

4. How can we be sure?
• We would like to check our assumptions for a « general
case » (intension)
– This can only be done through formal verification of the code
– This cannot be automatized for Turing-complete languages (cf.
halting problem)
• The alternative is then to generate explicitly all possible
configurations (extension) and test each of them
– But the number of configurations can be huge or infinite
• Now if our code passes the test on 100 randomly chosen
examples, we might still get some good level of confidence
in our implementation
– This is just what ScalaCheck proposes

5. Main idea
In ScalaCheck, we check properties on random
datasets created by generators.

6. Properties
• Logical statements that the function must satisfy, e.g.
Math Scala
 i  , 2*i == i+i forAll((i:Int) => 2*i == i+i)
exists((i:Int) => 2*i == 2)
 i  , such that 2*i==2 forAll{(i:Int,j:Int)=>
 i,j  , (i*j) == 0  i == 0 || j == 0 (i*j)==0 ==> (i==0 || j==0)}
Try those, you might have some surprises…
• Elements
– Quantifier: forall, exists, atLeast, iff, imply…
– Assertion: boolean function
• org.scalacheck.Prop

7. Generators
• org.scalacheck.Gen
• Basically a function Params => Option[T]
– Params: size + random number generator
• Built-in generators
– choose: integer in range
– oneOf : element from sequence of T or Gen[T]
– frequency : oneOf taking weights into account
– arbitrary : arbitrary instance (incl. edge cases)
• Gen is a Monad!
– map, flatMap, filter

8. Generator example
• Suppose we have
case class Person(name:String,age:Int)
• We can construct a Gen for Person
val personGen = for (n<-arbitrary[String];a<-
choose(1,125)) yield Person(n,a)
• We can test it using the sample function
personGen.sample
Not «?», but Unicode!
res: Option[Person] = Some(Person(???????????????,85))
• We can use it to test properties on Person
Prop.forAll(personGen)((p:Person) => p.age>0)
• If we want to use it implicitly we can declare it as an
arbitrary generator
implicit val arbPerson = Arbitrary(personGen)
Prop.forAll((p:Person) => p.age>0)

9. Generator example (cont’d)
• Now if we only want young people
val youngPeople = personGen.filter(_.age<31)
• We can use it for some specific tests
Prop.forAll(youngPeople)((p:Person) => p.age<50)
• If we want a list of 3 young people, we can
write
val trio = Gen.listOfN(3,youngPeople)
trio.sample
res: Option[List[Person]] =
Some(List(Person(????????????????????????,4), Person(
???????????????,9), Person(???????????,20))))

10. Complex failures
• Suppose you have a property that relies on
complex data (e.g. a list)
(l:List[Int]) => l.size==l.distinct.size
• Now this obviously fails for list with duplicates
• ScalaCheck could give the first failure
! Falsified after 5 passed tests.
> ARG_0: List("-2147483648", "1", "1933529754", "-726958561", "-
2147483648”, "750300922", "841716922", "-
2147483648", "1671473995")
• But it gives instead
! Falsified after 8 passed tests.
> ARG_0: List("-1", "-1")

11. How did this happen?
• ScalaCheck applies shrinking strategies to
gradually simplify the problem to a simplest
failing case
• Standard shrinking strategies are provided for
most common cases
– List
– Tuple
– String
– Int
– Container

12. Integration with Scala frameworks
• ScalaTest
– prop.Checkers trait
• check method
(but cannot use the should/must matcher
syntax)
• Specs2
– check function

13. Example
• Scampi class AlgebraTest extends FeatureSpec with Checkers with Algebra {
– Library for // [Generators for E…]
def associativity(op: (E, E) => E) =
(a: E, b: E, c: E) => simplify(op(op(a, b), c)) == simplify(op(a, op(b, c)))
Operations
Research def commutativity(op: (E, E) => E) =
(a: E, b: E) => simplify(op(a, b)) == simplify(op(b, a))
– Abstract algebra feature ("Addition") {
using GADTs val addition = (e1: E, e2: E) => e1 + e2
scenario ("scalars") {
– Very }
check((i: Int, j: Int) => Const(i) + Const(j) == Const(i + j))
mathematical scenario ("associativity") {
check(associativity(addition))
– Perfect fit }
scenario ("commutativity") {
check(commutativity(addition))
}
}
//…
}
hg clone https://bitbucket.org/pschaus/scampi

14. Other nice stuff
• collect to store generated values
• classify to make histograms
• Stateful testing
– Finite State Machines
– Types
• State
• Command

15. Pros
• Enhance readability
– Very declarative, appeals to mathematically-inclined
– Domain experts can read and comment
• Focuses on properties of input and output
– Instead of concrete examples
• Automatically explores edge cases
– Instead of tedious, incomplete and error-prone code
• Shrinks to simplest failing examples
Domain knowledge Active documentation

16. Cons
• It can be difficult to
– Find « good » properties
• Not trivially satisfied Reflection on the domain
• Not too complex to observe
– Create « good » generators
• Relevant edge cases Reflection on the specifications
– Write checking functions
• Implementing the check might be as error-prone as writing the function!
Focus on variations/set of properties
• Requires functional code
Good programming practice
• Beware of shrinking
– If your property relies on the size of your instances

17. Credits
• Inspiration
– Haskell library QuickCheck (Koen Claessen, John Hughes)
• Main contributors
– Ricky Nilsson
– Tony Morris
– Paul Phillips
– Yuvi Masonry
• Links
– GitHub: https://github.com/rickynils/scalacheck
– User guide:
http://code.google.com/p/scalacheck/wiki/UserGuide
– Articles:
http://code.google.com/p/scalacheck/wiki/ScalaCheckArticles

18. Thank you!