Video and slides synchronized, mp3 and slide download available at http://bit.ly/1cghaiX. Mat Wall describes some of the tools & techniques that are used within the UK Government Digital Service to try and make the government behave less like an enterprise and more like a startup.Filmed at qconlondon.com. Mat Wall is a technical architect who can still actually make things and now works for the Cabinet Office, a rare thing indeed. Mat works on elements of the GOV.UK website, as well as working within departments on transactional services. Twitter: @matwall

1. Green shoots in the brownest field
Mat Wall (@matwall)
Government Digital Service
https://www.gov.uk
Being a startup in government
Friday, 8 March 13

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/UK-gov-gds-tools-techniques

3. Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. GDS
Government development skills largely outsourced in 80s
Government IT vendor driven, procurement focussed
People who can write code (not COBOL) very rare
Friday, 8 March 13

5. GDS set up about 18 months ago
Bring digital capability & vision into government
Stimulate economy of smaller vendors
A startup within government
Friday, 8 March 13

6. Initially build alpha of GOV.UK
Now fully released into production
Migrating government department sites
Beginning to work on transactions
Friday, 8 March 13

7. Design Principles
1. Start with user needs
2. Do less
3. Design with data
4. Do the hard work to make it simple
5. Iterate wildly
6. Build for inclusion
7. Understand context
8. Build services, not websites
9. Be consistent, not uniform
10. Make things open: it makes things better
https://www.gov.uk/designprinciples
Friday, 8 March 13

8. reboot the government
Friday, 8 March 13

9. GOV.UK
Friday, 8 March 13

10. Quick answers
Friday, 8 March 13

11. Quick answers
Friday, 8 March 13

12. Quick answers
Friday, 8 March 13

13. Detailed guides
Friday, 8 March 13

14. Detailed guides
Friday, 8 March 13

15. Transactions
Friday, 8 March 13

16. Transactions
Friday, 8 March 13

17. Department sites
Friday, 8 March 13

18. Department sites
Friday, 8 March 13

19. Light weight architecture & development
Focussed on user needs
Good example of “How to do it” across
government
Friday, 8 March 13

20. Strategic architecture
Friday, 8 March 13

21. Very common in government IT
Small number of in house architects
Often not hands on
Almost all development outsourced
Friday, 8 March 13

22. Lay down ground rules
Select technologies up front
Often vendor focussed / led
Friday, 8 March 13

23. Attempt to:
reduce risk
minimise cost
produce predictable results
Friday, 8 March 13

24. Insert strategic architecture here
Friday, 8 March 13

25. Friday, 8 March 13

26. ESB
Friday, 8 March 13

27. ESB
ESB
Friday, 8 March 13

28. ESB
ESB
Friday, 8 March 13

29. ESB
ESB
OPA
Friday, 8 March 13

30. ESB
ESB
OPAEDM
Friday, 8 March 13

31. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Friday, 8 March 13

32. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
Friday, 8 March 13

33. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
Friday, 8 March 13

34. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Friday, 8 March 13

35. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13

36. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Now, what shall we make?
Friday, 8 March 13

37. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Imagine being a developer here
Most decisions made for you
Little room for innovation
Friday, 8 March 13

38. reduce risk = no innovation, create lock in
minimise cost = ha ha
predictable results = slow delivery
Friday, 8 March 13

39. Platforms
Friday, 8 March 13

40. Platforms first = bottom up architecture
We want to focus on user needs first
Friday, 8 March 13

41. Most of our apps start out as a rough idea
on someones laptop, not as a platform
Friday, 8 March 13

42. Platforms emerge by constant delivery
Friday, 8 March 13

43. Space
Friday, 8 March 13

44. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Web routing
Provisioning
Configuration management
Services / libraries
Persistence
Monitoring
Friday, 8 March 13

45. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Ruby / rails / sinatra/ jekyll
Scala / play 2.0
Java / dropwizard
MongoDB, MySql, ElasticSearch, Redis
nginx, varnish, Akami
puppet
statsd / graphite / nagios / ganglia
funkload
Friday, 8 March 13

46. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
Space
Friday, 8 March 13

47. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend
inside gov
smart
answers
licensingsearch
Content API
Publisher
Panopticon MongoDB MySQL
CMS tools
Licensing
admin
Sign onneed-o-tron
Friday, 8 March 13

48. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13

49. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Content model split across apps
Complex interaction between apps
Brittle, what if interaction fails?
Friday, 8 March 13

50. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13

51. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API
Publisher
PanopticonMongoDB
need-o-tron
routing
Friday, 8 March 13

52. Prototyping
ProvisioningConfiguration
Persistence
CDN, web & routing
Continuous integration Monitoring Alerting Performance test
Libraries/gems/jars/utils
Services
.gov.uk
frontend search
Content API /
Panopticon
Publisher
MongoDB
need-o-tron
routing
Event broadcast
Friday, 8 March 13

53. Space
Help developers create a safe space
Create small, simple apps in that space
Provide lightweight components & processes, and ability to create more
Don’t worry about trying to solve all of the problems up front
Allow for innovation & learning
Friday, 8 March 13

54. Momentum
Friday, 8 March 13

55. Momentum
We far prefer momentum to strategy
Make decisions as late as possible
Short iterations mean you can’t go to far wrong
Easy to put things back on track
Allows for innovation, experiment & learning
Friday, 8 March 13

56. New system: apply to register to vote (ERTP)
Initially seemed quite simple
Surely just a form and a database?
Real world is more complex
Friday, 8 March 13

57. Local authority 1 Local authority 2 Local authority 387....
Citizens
DWP
Friday, 8 March 13

58. Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWP
Web space
LA space
DWP space
Friday, 8 March 13

59. Citizen API
Local auth API
MatchingAPI
Local authority 1 Local authority 2 Local authority 387....
DWPMongoDB
Frontend web
Feeds Monitoring
Friday, 8 March 13

60. Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13

61. Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13

62. Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Friday, 8 March 13

63. Local auth API
MatchingAPILocal authority 1 Local authority 2 Local authority 387....
DWP
Frontend web
Anything goes to keep momentum
Friday, 8 March 13

64. Legacy / complex systems
Friday, 8 March 13

65. So far most of our systems have been very simple
Easy to be small, beautiful & agile
But that will change with transactions
Friday, 8 March 13

66. Government IT estate complex, old
Pre-web technologies
Oldest system I have found so far is 40 years!
How can we work in this environment?
How can we retain our momentum?
Friday, 8 March 13

67. ESB
ESB
OPAEDM
WEB WEB WEB WEB
Security stuff
XML translation layer
SAN SAP
Enterprise app servers
Friday, 8 March 13

68. Integration strategies:
Could rebuild old system.
Could export data from old system into new.
Use what we have learned to create our own space, with our own momentum
Friday, 8 March 13

69. Shiny application
Domain API Domain API
Create prototype application backed of simple REST domain APIS
Each API encapsulates a single domain concept
(ie: User profile, transaction history, application service)
Domain APIs filled with mock data
HTTP integration test suite produced
Domain APIDomain API
Friday, 8 March 13

70. Shiny application
Domain API Domain API
Integration tests form spec that can be shared between old & new world
Api built by new team, using old APIs / integrations
APIs can be built in existing system, by existing team
Or as a joint effort, using a facade
Domain APIDomain API
Friday, 8 March 13

71. Shiny application
Domain API
New space Old space
Friday, 8 March 13

72. Shiny application Domain API
New space Old space
http, JSON
GET /user/341353/transactions
select * from transactions where
userId=431353
Friday, 8 March 13

73. Shiny application
Domain API
New space Old space
database
Data feed
Friday, 8 March 13

74. Give developers space
then get out of the way
Momentum is king
Friday, 8 March 13

75. http://github.com/alphagov
http://digital.cabinetoffice.gov.uk/
https://www.gov.uk
Friday, 8 March 13