Technical operations is plagued with an unhealthy infatuation of typically untested, imperative code with a high reliance on shared mutable state using dynamically typed languages such as Ruby, Python, Bash, and - ugh - remember Perl? :) In an age where building reliable infrastructure to elastically scale applications and services are paramount to business success, we need to start rethinking the infrastructure engineer’s toolkit and guiding principles. This talk will take a look at applying various functional techniques to building and automating infrastructure. From functional package management and congruent configuration to declarative cloud provisioning we’ll see just how practical these techniques typically used in functional programming for applications can be used to help build more robust and predictable infrastructures. While specific code examples will be given, the emphasis of the talk will be on guiding principles and functional design.

1. Functional Operations
#dmconf1521 November 2015
Susan Potter
Lookout
twitter: @SusanPotter
github: mbbx6spp

2. % whoami
Figure: From developer to (dev)ops engineer

3. Agenda
1 Motivation for reasoning
2 Review functional programming 101
3 Illustrate forms of reasoning
4 Case study: Reimagine package management

4. Edsger Dijkstra

5. Reliability
“Those who want really reliable software will discover
that they must find means of avoiding the majority of
bugs to start with, and as a result the programming
process will become cheaper. If you want more effective
programmers, you will discover that they should not
waste their time debugging, they should not introduce
the bugs to start with.”[?]

6. Why care now?
1 Economic factors
necessity of distributed systems

7. Why care now?
1 Economic factors
necessity of distributed systems
2 Human factors
high churn/turnover, low quality of ops life

8. Why care now?
1 Economic factors
necessity of distributed systems
2 Human factors
high churn/turnover, low quality of ops life
3 Technological factors
programmable infrastructure & FP no longer just for academics

9. The Problem. . .

10. Application Delivery
1 Provision infrastructure
2 Configure nodes
3 Orchestrate services

11. Need to support more
1 Application services
2 Environments
3 Data services
4 Distributed services

12. Optimize for
1 Scalability solved by on-demand ”cloud”

13. Optimize for
1 Scalability solved by on-demand ”cloud”
2 Reliability solved by . . .

14. So what yields reliability?

15. Reason
The required techniques of effective reasoning are pretty
formal, but as long as programming is done by people
that don’t master them, the software crisis will remain
with us and will be considered an incurable disease. [?]

16. Functions 101

17. Functions have inputs (Ruby)
1 # Two input arguments here
2 def add(x, y)
3 x + y
4 end
5
6 # One input argument here
7 def len(s)
8 s.size
9 end

18. Functions have inputs (Scala)
1 object Functions {
2 // Two typed input arguments here
3 def add(x: Int , y: Int) = x + y
4
5 // One typed input argument here
6 def len(s: String) = s.size
7 }

19. Functions return a result
1 scala > add(5, 6)
2 res0: Int = 11
3
4 scala > len("Hello ,␣Barcelona")
5 res1: Int = 16

20. Only depend on inputs 1/2
1 scala > val defaultTimeout : Int = 30
2 scala > val timeout1: Option[Int] = Some (15)
3 scala > val timeout2: Option[Int] = None
4 scala > :paste
5 def defaulter[A](a: => A, ma: Option[A]) =
6 ma match {
7 case Some(x) => x
8 case None => a
9 }

21. Only depend on inputs 2/2
1 scala > timeout1
2 timeout1: Option[Int] = Some (15)
3
4 scala > timeout2
5 timeout2: Option[Int] = None
6
7 scala > defaulter(defaultTimeout , timeout1)
8 res0: Int = 15
9
10 scala > defaulter(defaultTimeout , timeout2)
11 res1: Int = 30

22. Return same result given same inputs
1 scala > len("Hello ,␣Barcelona")
2 res0: Int = 16
3
4 scala > len("Hello ,␣Barcelona")
5 res1: Int = 16
6 ...
7 scala > len("Hello ,␣Barcelona")
8 res3333333333: Int = 16
9
10 scala > // Always!

23. The Big idea
Referential Transparency
Given same inputs, return same result. Always.

24. Functions can use other values
1 // type aliasing a function
2 type Pred[A] = A => Boolean
3
4 // Passing a function as an input argument
5 def is[A](p: Pred[A])(a: A) = p(a)
6
7 // This uses already defined function +is+
8 def not[A](p: Pred[A])(a: A) = !is(p)(a)

25. Values can be functions
1 // Returning a function as a value
2 def lessThanN(n: Int): Pred[Int] = _ < n
3
4 // Using two in scope functions
5 def islessThanN(n: Int)(x: Int) =
6 is(ltN(n))(x)
7
8 // Those values can be functions :)

26. Another important idea
Higher Order Functions
Build useful functions from simpler functions!

27. Questions so far?
Figure: Awake?

28. Function Composition

29. UNIX Pipes 1/4
1 sh > echo -n "Hello ~~~"
2 | sed ’s/~//g’
3 | tr ’[: lower :]’ ’[:upper :]’
4 | wc -c
5 5

30. UNIX Pipes 2/4
1 sh > alias sanitize=’sed "s/~//g"’
2 sh > alias toUpper=’tr "[: lower :]" "[: upper :]"’
3 sh > alias len=’wc -c’
4
5 sh > alias myfun0=’sanitize | toUpper ’
6 sh > alias myfun1=’myfun0 | wc -c’

31. UNIX Pipes 3/4
1 sh > echo -n "Hello ~~~"
2 | sed ’s/~//g’
3 | tr ’[: lower :]’ ’[:upper :]’
4 | wc -c
5 5
6
7 sh > echo -n "Hello ~~~" | myfun1
8 5

32. UNIX Pipes 4/4
• Character-based
• File descriptors

33. Function Composition 1/3
1 def toUpper = (s: String) => s.toUpperCase
2 def len = (s: String) => s.size
3 def sanitize = "~".r replaceAllIn (_, "")

34. Function Composition 2/3
1 scala > def myfun0 = sanitize andThen toUpper
2 scala > def myfun1 = myfun0 andThen len
3
4 scala > myfun0 "Hello ~~~"
5 res0 = HELLO
6
7 scala > myfun1 "Hello ~~~"
8 res1: Int = 5

35. Function Composition 3/3
• Value-based
• Functions

36. Questions so far?
Figure: Awake?

37. Functions as building blocks
Figure: Build solid walls/foundations from simple, regular building blocks

38. Reasoning

39. Equational Reasoning
• Simpler testing; reduce test suite complexity
• Substitute repeated expression with name
• Refactoring is trivial; not error prone

40. Testing non-RT code
Figure: Side effecting, non-RT code requires complex testing setup.

41. Equational Reasoning
1 scala > f(v1 , v2 , otherfun) == expected
2 res0: Boolean = true
3
• no setup/teardown complexity to test assertions
• no mutation of SUT just for testing

42. Equational Reasoning
1 scala > val x = "YOLO"
2 x: java.lang.String = YOLO
3
4 scala > val r1 = x.reverse
5 r1: String = OLOY
6
7 scala > "YOLO".reverse
8 res0: String = OLOY
9

43. Equational Reasoning
1 scala > var age = 27
2 scala > def incr = { age += 1; age }
3
4 scala > incr
5 res0: Int = 28
6
7 scala > incr
8 res1: Int = 29
9

44. Axiomatic Reasoning
• Axioms are basic assumptions
• Allows us to specify properties
• Infer potential problems due to properties
• What can we guarantee? What can we not?

45. Axiomatic Reasoning
1 class Vpc < AwsResource
2 def initialize(conf)
3 @cidr = conf[’vpc.cidr ’]
4 end
5 end
6 # Is the Vpc instance created with an
7 # empty conf Hash guaranteeing the
8 # premise of the initialize contract?
9

46. Axiomatic Reasoning
1 // validate inputs before instantiating
2 case class Vpc(cidr: Cidr)
3
4 def createVpc(ipStr: String , pfx: String) =
5 for {
6 ip <- IpAddress.toIpAddress(ipStr)
7 prefix <- Cidr.toPrefix(pfx)
8 } yield Vpc(Cidr(ip , prefix ))
9

47. Axiomatic Reasoning
1 // if fun is 1-to -1 or bijective
2 // we know inverse must exist
3 def toLower(u: UpperCaseChar ): LowerCaseChar = ??
4 def toUpper(l: LowerCaseChar ): UpperCaseChar = ??
5
6 // idempotency typically important in ops
7 f(f(x)) == f(x)
8

48. Axiomatic Reasoning
1 // commutativity important in dist sys
2 // f, binary operator over type A
3 // x and y are of type A then
4 // x @ y = y @ x
5 f(x, y) == f(y, x)
6
7 // associativity important in concurrency
8 // f, binary operator over type A
9 // x, y, and z are of type A then
10 // x @ (y @ z) = (x @ y) @ z
11 f(x, f(y, z)) == f(f(x, y), z)
12

49. Axiomatic Reasoning - Distributed
Systems
• Simple causal consistency (partial ordering)
• CRDTs (semilattice algebra)
• Invariant confluence (coordination free
execution)

50. Axiomatic Reasoning - Concurrency
• MVars, TVars, LVars have algebraic properties
• Design for liveness via axiomatic reasoning

51. Axiomatic Reasoning - More Exhaustive
Testing
• Property-based testing
1 // untested - uses scalacheck
2 def inverse[A]( implicit a: Action[A]) =
3 Prop.forAll { (x: A) =>
4 x === a.inverse(a.inverse(x))
5 }
6
7 implicit def actionArbitrary [A]
8 (implicit a: Arbitrary[A]) = Arbitrary { /*
9

52. Generic Reasoning
• Reasoning on generic type functions
• Less specific the types the more we know about
the function
• Find design-time bugs

53. Generic Reasoning
1 def f0[A](a: A): A = ???
2

54. Generic Reasoning
1 // Only definition
2 def f0[A](a: A): A = a
3

55. Generic Reasoning
1 def f1[A](a: A)(ma: Option[A]): A = ???
2

56. Generic Reasoning
1 def f1[A](a: A)(ma: Option[A]): A = a
2
1 def f1[A](a: A)(ma: Option[A]): A = ma match {
2 case Some(x) => x
3 case None => a
4 }
5

57. Generic Reasoning
1 def f2[A](as: List[A]): A = ???
2

58. Generic Reasoning
1 // what about an empty list? Yikes!
2 def f2[A](as: List[A]): A = ???
3

59. Generic Reasoning
1 // Assuming we want the head element!
2 // A more sensible type signature design
3 def f3[A](as: List[A]): Option[A] = as match {
4 case Nil => None
5 case a :: rest => a
6 }
7

60. Generic Reasoning
1 def f4[A](ma: Option[A]): A = ???
2

61. Review

62. Reasoning Process
• Assume as little as possible
• Define all inputs
• Build only on top of RT
• Derive properties from domain
• Infer properties from generic types

63. Methods / Techniques
• Encode normal error cases in results
• Strive for termination
• Strive for RT where ever possible

64. Iteration / Improvement
• So much to learn
• Start out informal; add formal reasoning as
needed/learned
• The more we do the easier it gets

65. Example in the wild

66. Mainstream Package Management
Based on shared + mutable state (filesystem)

67. Violates RT

68. Alternatives
• shared + immutable
• private + mutable
• expensive coarse grained locks
• hybrid without the expense

69. Define all inputs
• Force clean build env (chroot)
• Requires explicit inputs
• Full dependency definition

70. Ensure RT
• Use private mutable space
• Different inputs, different result
• Symlink unique results (atomic op)

71. Different inputs give unique results
• Shared + immutable provides RT
• New inputs result in different path
• Install many versions/configurations
• Means we can do binary substitution

72. RT foundation
• Update existing node more reliably
• Supports immutable infrastructure
• Supports congruent configuration

73. Recap
1 Referential Transparency allows us to reason
3 HOFs & Composition is the motar
4 Logical reasoning provides solid foundation

74. Recap
1 Referential Transparency allows us to reason
2 Functions are simple, regular building blocks
3 HOFs & Composition is the motar
4 Logical reasoning provides solid foundation

75. Don’t fear the maths!
Figure: Domain specific language of the sciences and engineering

76. Questions
Figure: Heckle me @SusanPotter later too.