2. • Who are First Utility
• Creating a high availability platform
• Enabling customer control and insights through analytics
• Providing a self healing platform with event history
• Monitoring and alerting across the platform
• Conclusion
Contents
4. First Utility
● Largest independent energy supplier
● Supply almost 1 million UK homes
● One of the first to provide Smart meters
● Provide broadband and home services as well
● Technology driven company
6. Creating a high availability platform
● National smart meter rollout
○ Aim is to have all 53 million residential premises in
England, Wales, Scotland with Smart Meters by 2020
● Energy suppliers are required to offer smart meters to all
their customers
● Our immediate need to manage a high volume of meter
reads with high availability
● Meter access devices can potentially provide data every 5-
10 seconds from the total meter estate
7. Creating a high availability platform
● Smart reads ideal for an event sourced type database
● Saw the need to add resilience, high volume and linear scaling capability into
our key information base
● Opportunity to manage business expectation through service layers
● Providing a meter agnostic business layer
“One of the things DataStax offered to us was the ability to really compress that learning curve. So we had
some of their key experts come in and spend a few weeks with us over a period of time and provide that
technical leadership” Bill Wilkins, CIO, First Utility.
http://diginomica.com/2017/02/02/first-utility-makes-smart-meters-even-smarter-datastax-platform/
8. Creating a high availability platform
store
From Monolithic to Microservices Architecture
Application Server
store
application
(instance 1)
application
(instance 1)
application
(instance 1)
Docker Container
application
(instance 2)
Application Server
Docker Container
application
(instance 1)
application
(instance 2)
single node
rdms
multiple node
distributed store
9. Creating a high availability platform
Meter Type
2
Meter Type
1
Customer
Insights
Billing
Industry
Data
Warehouse
Meter Type
n
spark
Usage View
Billing View
Industry
View
Data Stores
spark
Meter Data 1
Meter Data 2
Meter Data 3
spark
10. Creating a high availability platform
● Multiple DCs
● Appropriate
RF
● DCs split
between
Business
Critical access
and Analytics
12. Enabling Customer Control and Insights
● Provide customers with an insightful view
● Help customers better manage their usage
● My energy comparison of usage
● Allow control of read submission frequency
● In home display reflects current tariff and usage
13. Enabling Customer Control and Insights
Meter
Customer
Insights
Meter
Preferences
Meter Data
Usage Viewread
preference/price TariffIHD
Data Storage
read
18. Providing a self healing platform
● Record commands sent to the meter
● Detect failures
● Replay commands (with retry limit)
Meter
commands
meter state
reconcile
replay
spark
Event
command
response
20. Monitoring and alerting across the platform
● Eyes and ears
● Dashboards of platform behaviour
● Operational: is the platform working
● Business information: reporting and analytics
23. Conclusion
● Gone from read only system to ability control of meter (read/write)
● Self healing
● Real time monitoring and alerting
● Enduring solution can scale both processing and persistence
● Lessons learnt
○ Denormalise - many copies with different PK
○ Know your queries in advance
○ Reporting and aggregation - duplicate, spark or counters