This slide deck describes the most successful software project of my career: automating a 40-person human compiler yielding a reduction in project time from months to hours and cost from $1,000,000 to $100s for our client. The system is still in operation years later and has never had a single bug filed against it.

1. Jon Harrop, Flying Frog Consultancy

2.  Jon Harrop
 Started programming in 1981
BASIC, Pascal, 6502, ARM, C, C++
 At British Petroleum in 1995
UFI
 Physics and Chemistry at Cambridge
Fortran, SML, OCaml, Mathematica, F#
 Co-founded Flying Frog in 2005
◦ Sell books, software and consultancy services
◦ Over 1,100 clients in industry using OCaml & F#

3.  One of the world’s largest insurance
companies.

4.  One of the world’s largest insurance
companies.
 Actually, almost everyone else too…

5. 1. An ordinary delivery process
2. Streamlining the delivery process
3. Technical aspects of the solution

6.  Automate repetition
◦ Traders in a pit
◦ Actuaries with calculators
 Use computers!

7.  Much better performance
◦ Latency
◦ Throughput
 More sophisticated algorithms

8. 1. Business idea!

9. 1. Business idea!
2. Model idea: MS Excel.

10. 1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.

11. 1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.
4. Test idea: NUnit (feedback to 2)

12. 1. Business idea!
2. Model idea: MS Excel.
3. Build idea: C++/Java/C#.
4. Test idea: NUnit (feedback to 2)
5. Deploy production code.
Room for improvement?

13. Room for improvement in:
 Classic waterfall approach.
◦ Iterations are an absolute killer.
 Delivery speed and cost.
◦ This is a long journey.
So what does an iteration look like?

14.  First iteration highlights issues.
◦ Civilized discussion.

15.  First iteration highlights issues.
◦ Civilized discussion.
 Second iteration, more issues.
◦ Stressful disagreements.

16.  First iteration highlights issues.
◦ Civilized discussion.
 Second iteration, more issues.
◦ Stressful disagreements.
 Later iterations.
◦ Wild fighting.

17. Big problems:
 Accuracy.
◦ Excel/Word are too free-form.
 Complexity.
◦ Models are complicated, lots of dispatch.
 Latency.
◦ Complected testing.

18. Old process:

20.  See the forest for the trees.
◦ Don’t just rewrite code in language du jour!!!
◦ Common mistake when adopting new technologies.
 Overhaul the delivery process…

21. A new process:
Observations

22. Observations
 Separate people/jobs
◦ One person uses Tool to turn ideas into products.
◦ Another person maintains Tool.
 Separate tests
◦ Tests for the model.
◦ Unit test for the Tool.
Advantages

23. Advantages:
 Business people ship products themselves!
 Less contention
◦ Faster iterations
◦ Quicker time-to-market
◦ Cheaper delivery
◦ Happier employees
 Scalable delivery process
◦ Add more tool users
◦ Add more tool developers

24. Challenges:
 Designing bespoke Tool
◦ Hard to gather requirements:“Define the language
you dream in?”
 Training
◦ Users
◦ Developers
 Plumbing
 Deployment

25. Build a new tool for business users that:
 Has a great user experience.
◦ Graphical application.
◦ Interoperates with Excel.
 Creates precise specifications.
◦ Catch errors as early as possible.
 Mirrors the production environment.
◦ Code reuse.
 Integrates testing.
 Is tuned to their needs.

26. DSLs have a rich history but GUIs are notably
absent…
DSL Patterns by Martin Fowler

27. No need to ship a DSL that looks worse than
this:

28. No need to ship a DSL that looks worse than
this:
1974

29. Started with a working demo
 Developed in one week!
 Slick looking
 Easy to use
 Bespoke language
 Integrated testing
 Their own worked examples included
 Demo runs on their machines (WPF)

30. Final solution:
 Development effort
◦ 3 months to design
◦ 3 months to build
 Code size
◦ 10kLOC of F#
 Performance
◦ 20,000 msg/s
◦ Under 35ms door-to-door latency

31. A different 3 month project:
 114µs latency.
 200,000 tps.
 With fault tolerance!
 Async message passing over Infiniband
 Low-latency allocationless F# code

32.  Training
◦ On-the-fly
◦ 30 people
◦ 5 teams
 Emergency
◦ Valuable change came in late
◦ Estimated 9 months and £1m
◦ Completed in 24 hours
 Maintenance
◦ All done by client now
◦ After one week of F# training
◦ Ordinary developers

33. “Pattern matching is quite neat, as is the partial
application of functions”
“rapidly coming to appreciate the benefits
when it comes to refactoring code”
“Our plans are getting bigger each week as we
can see more and more flexibility, it’s great!”

34. Benefits
 Delivery speed
◦ From idea to production 100x faster
 Delivery costs
◦ Maybe 10,000x lower?!

35.  F# is good for programmatic GUIs
 Approach described in the F# Journal
 Custom Control union type
 Compiled to WPF Control
 Custom Editable Control
 Reuse for editable strings, unions etc.
 Compose into a UI tree
 Peek Value property to get the “rules”

37. Nice name!

38. Input
Variables

39. Patterns

40. Output
Variables

41. Expressions

42. Take a leaf out of Excel:
 Editable content:
 Display version:

43. Display version:

44. Editable content:
A bit like type checking…

45. type Control =
| Gap
| Label of string
| Title of string
| Dock of (Position * Control) list
| Ctrl of FrameworkElement
| …

46. let rec make : Control -> FrameworkElement = function
| Gap ->
let ctrl = new FrameworkElement()
ctrl.HorizontalAlignment <- HorizontalAlignment.Stretch
ctrl.VerticalAlignment <- VerticalAlignment.Stretch
ctrl
| Label s -> upcast Controls.Label(Content=s)
| Title s -> upcast Controls.TextBlock(Text=s, FontSize=24.0)
| Dock xs ->
let panel = Controls.DockPanel()
for p, ctrl in xs do
let ctrl = make ctrl
Controls.DockPanel.SetDock(ctrl, p.AsWPF)
panel.Children.Add ctrl |> ignore
upcast panel
| …

47. let ui =
Dock
[ Top, Title "Bespoke business rules engine"
Bottom,
Tabs
[ Label "Inputs",
Dock[ Top, Label "This version was…"
Bottom, Scroll inputs ]
Label "Rules",
Dock[ Top, Label "The following rules…"
Bottom, Scroll(Ctrl rulesView) ]]]

48. let StringValue(value: Value) =
let label = Controls.Label()
let textbox = Controls.TextBox()
let setContent = function
| VString text ->
label.Content <- text
textbox.Text <- text
| _ -> failwith "Invalid value"
let ctrl =
Editable<Value>(value, label, textbox, setContent)
textbox.TextChanged.Add(fun _ ->
ctrl.SetValue(Some textbox.Text
|> Option.map VString))
ctrl

49. let UnionValue(allCases, value: Value) =
let label = Controls.Label()
let combo = Controls.ComboBox()
let itemOf case = case, Controls.ComboBoxItem(Content=case)
let items = Map[for case in allCases -> itemOf case]
let setContent = function
| VCase case ->
label.Content <- case
combo.SelectedItem <- items.[case]
| value -> failwith "Invalid value"
let ctrl = Editable<Value>(value, label, combo, setContent)
combo.ItemsSource <- [ for KeyValue(_, ctrl) in items -> ctrl ]
for KeyValue(case, x) in items do
x.Selected.Add(fun _ -> ctrl.SetValue(Some(VCase case)))
ctrl

50. The expression language:
type Expr =
| EBool of bool
| ENumber of decimal
| EString of string
| EDate of System.DateTime
| ETuple of Expr list
| EVar of string
| …

51. The expression interpreter:
let rec eval vars expr =
match expr with
| EBool b -> VBool b
| ENumber x -> VNumber x
| EString s -> VString s
| EDate d -> VDate d
| ETuple fs -> VTuple(List.map (eval vars) fs)
| EVar v ->
match Map.tryFind v vars with
| None -> failwithf "Unknown variable '%s'" v
| Some x -> x
| …

52. Challenges
 Slow startup times
 Poor GUI performance (grid of numbers)
 Minor perf bugs
 Brittle .NET serialization

53. Startup times
Problems:
 64-bit NGEN bug
 Poor WPF Grid performance
Solutions:
 Use 32-bit client
 Custom WPF control…

54. Startup times
 Took two days to fix
 Startup times down from 15s to 0.1s

55. GUI performance
Problem:
 The WPF Grid control is:
◦ very slow to start up
◦ very slow to scroll
Solution:
 Custom FastGrid control…
 Low-level rendering in F#
◦ Calls DrawGlyphs

56. GUI performance
 16x faster startup than vanilla WPF.
 Solution in the F# Journal.
 Still a lot of room for improvement…

57. Minor perf bugs
 sprintf “%A”
 Decimal slow (but ok)

58. Serialization
Problem:
 .NET serialization not great for F#.
 Bloated format (includes types).
 Brittle when loading (incompatible types).
Solution:
 Custom structural serialization library.
 Handles values of all F# types.
 Solution in the F# Journal.

59. Serialization
write true = "<bool>True</bool>"
write {anInt=2; aReal=3.4m} = "<record><
field><name>anInt</name><value><int>2</i
nt></value></field><field><name>aReal</n
ame><value><decimal>3.4</decimal></value
></field></record>"

60. Structurally-typed serialization handles:
 unit
 bool
 byte, int, float, decimal
 System.DateTime
 string
 Tuples, records, unions
 Lists, arrays, sets and maps

61. Any questions, please contact me:
Jon Harrop <jon@ffconsultancy.com>