Summary of transaction log prototype for GOV.UK publishing platform

1. David Heath
Publishing platform team
Government Digital Service
GDSPublishing platform team

2. Transaction log prototype
2GDSPublishing platform team

3. Why?
3GDSPublishing platform team

4. Complexity
4GDSPublishing platform team

5. many bugs...
5GDSPublishing platform team
* many bugs and issues which we’ve experienced have been related to that

6. It's a master copy
of every change or
transaction
6GDSPublishing platform team

7. It’s an audit log
7GDSPublishing platform team
a feature we’d really like to have

8. Future-proof
8GDSPublishing platform team
if we have a log of every operation, it’s easy to adapt to possible future requirements, because we can easily add new “derived representations”

9. What is a derived
representation?
9GDSPublishing platform team

10. Bank account
10GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
Balance: £50

11. Bank account
11GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
Balance: £45.01

12. Bank account
12GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
3/9/2015 Cheque 003 £20
Balance: £65.01

13. Bank account
13GDSPublishing platform team
Date Description Amount
1/9/2015 Initial deposit £50
2/9/2015 Pod Ltd -£4.99
3/9/2015 Cheque 003 £20
Balance: £65.01
Derived representation

14. In publishing…
GDSPublishing platform team
Event Log

15. In publishing…
GDSPublishing platform team
Event Log
Current documents

16. In publishing…
GDSPublishing platform team
Event Log
Current documents
History of published documents

17. In publishing…
GDSPublishing platform team
Event Log
Current documents
History of published documents
Editorial history and comments

18. GDSPublishing platform team
Event Log
Current documents
History of
published
documents
Editorial history
and comments
Admin elasticsearch index
Message on message queue (asynchronous update)
(separate service/data store)
(Synchronously updated)Publishing API
* Derived representations may be in separate systems
* Some things need to be updated synchronously, those would need to be in the same data store and updated using a database transaction to guarantee consistency.

19. What we did in the prototype
19GDSPublishing platform team

20. What we did in the prototype
• event log
20GDSPublishing platform team

21. What we did in the prototype
• event log
• commands (for modification)
21GDSPublishing platform team

22. What we did in the prototype
• event log
• commands (for modification)
• queries (read from derived
representation)
22GDSPublishing platform team

23. What we did in the prototype
• event log
• commands (for modification)
• queries (read from derived
representation)
• wired up to policy publisher
23GDSPublishing platform team

24. 24GDSPublishing platform team
we used postgresql because it’s awesome
- real transactions
- JSON data type

25. Show me some code!
25GDSPublishing platform team

26. Remaining questions
26GDSPublishing platform team
• conceptual model of editing/
workflow

27. Remaining questions
27GDSPublishing platform team
• conceptual model of editing/
workflow
• how much validation?
• use schemas?

28. Remaining questions
28GDSPublishing platform team
• conceptual model of editing/
workflow
• how much validation?
• use schemas?
• write performance

29. Remaining questions
29GDSPublishing platform team
• conceptual model of editing/
workflow
• how much validation?
• use schemas?
• write performance
• replay, snapshotting etc

30. David Heath
Publishing platform team
Government Digital Service
GDSPublishing platform team