Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Real-time Financials 

with Microservices and

Functional Programming
Vitor Guarino Olivier

vitor@nubank.com.br
@ura1a

h...
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the sprea...
MAIN PRODUCT
Live since September 2014
A TECHNOLOGY DRIVEN APPROACH TO
FINANCIAL SERVICES
CONTINUOUS DELIVERY
MICROSERVICES
INDEPENDENTLY AND CONTINUOUSLY
DEPLOYABLE
DECOUPLED AND EASY TO REPLACE
BOUNDED BY CONTEXT AND INDEPENDENTLY
DEVELOPED
WHAT HAPPENS WHEN WE NEED TO
COMBINE DATA ACROSS SEVERAL SERVICES?
ESPECIALLY IN REAL-TIME
- Running on AWS, 2 AZs, config as code, 

immutable infra, horizontally scalable,

sharded by customers
SERVICE ARCHITECT...
DATOMIC
- Immutable, append-only database
- ACID on writes (atomic, consistent, isolated, durable)
Lucas Cavalcanti & Edwa...
The Problem
WE HAVE OVER 90 SERVICES
THE PROBLEM:

A LOT OF BUSINESS LOGIC DEPENDS ON DATA
ACROSS MANY SERVICES
Should I authorize a purchase? Should I block a...
THE PROBLEM:

WE ARE SHOWING THESE NUMBERS TO THE
CUSTOMER IN REAL TIME
THE PROBLEM:

NO CANONICAL DEFINITION OF OUR KEY
NUMBERS
- Ad-hoc definitions created by analysts and engineers
- Analysis...
A BALANCE SHEET IS THE CANONICAL WAY
OF REPRESENTING FINANCIAL INFO
- We can apply generally accepted accounting principle...
THE MODEL
- Entry: represents a debit and a credit to two book-accounts
- Meta-entity: it’s a reference to the external en...
Double-entry accounting

service
OUR GOAL FOR OUR ACCOUNTING LEDGER

(aka DOUBLE-ENTRY SERVICE)
- High availability to other services, clients, and analyst...
THE IDEAL FLOW
f(payload)
MOVEMENT[ ]
EVENT
ACID transaction
- Event ordering doesn’t matter
- No mutable state
- Needs to...
{:purchase

{:id (uuid)

:amount 100.0M

:interchange 1M

:post-date "2016-12-01"}}
Initial Balances: 

Current Limit R$ 1...
WE CAN'T GUARANTEE CONSISTENCY,

BUT WE CAN MEASURE IT
f(payload)
MOVEMENT[ ]
ACID transaction
- Kafka Lag
- Service downt...
PURE FUNCTIONS OF THE PAYLOAD
WON'T ALWAYS WORK
The Stateful Flow
{:payment

{:id (uuid)

:amount 150.00M

:post-date "2016-12-01"}}
[{:entry/id (uuid)

:entry/amount 100.0M

:entry/debit-...
{:payment

{:id (uuid)

:amount 150.00M

:post-date "2016-12-01"}}
[{:entry/id (uuid)

:entry/amount 100.0M

:entry/debit-...
THE STATEFUL FLOW
- Movements in the past will modify all future balances
- Adapters are a function of the event payload A...
INVARIANTS
- Some balances can’t coexist (no late alongside prepaid)
- We can establish invariants that must hold true at ...
THE STATEFUL FLOW
EVENT
f(payload, )
ACID transaction
f( )
VALID STATE?
FIX VIOLATION
INVARIANT VIOLATIONS?
NO
MOVEMENT[ ]...
CHALLENGES
CHALLENGES
- Fixing invariants logic is extremely complex.
- Datomic indexing is tested until 10 billion facts.
- Datomic ...
GENERATIVE TESTING
- We generate random events from our schemas (bill, purchases, payments, etc)
- Write a function that d...
GENERATIVE TESTING
(def balances-property
(prop/for-all [account (g/generator Account)

events (gen/vector (gen/one-of [(g...
MONITORING / REPLAY HISTORY TOOLING
- Other services have republish endpoints 

(same payload and meta data as original th...
SHARDING BY CUSTOMER / TIME
- No cross customer entries allows for per customer sharding
- We shard the database by time f...
ETL
extract logs
facts to table

(one per
entity type)
tables stored
applies functions to generate balances
balances
on re...
The Result
Text placeholder
2015-01 2015-04 2015-07 2015-10 2015-01 2016-03
REAL TIME BALANCE SHEET
2 TIMELINES
ACTUAL (DB) TIME
audit trail / Datomic log
“when did we know”day 0 day 30 day 90
BUSINESS TIME
official versio...
WHAT WE LIKE
- Canonical definition of our most important numbers
- Financial analysis applied at a the customer level in ...
42
nubank.com.br/jobs
vitor@nubank.com.br
@ura1a
https://gist.github.com/ura1a to get snippets of our domain!
THANK YOU!
Watch the video with slide synchronization on
InfoQ.com!
https://www.infoq.com/presentations/nubank-
financial-systems
Prochain SlideShare
Chargement dans…5
×

Functional / Microservices in Real-Time Financials

552 vues

Publié le

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2z0lTca.

Vitor Olivier presents how Nubank has built the system of record based on functional programming principles, the tools they used (Clojure, Datomic, Kafka), the challenges they faced when taking it to scale, and the benefits of their approach, including data science modeling, real-time customer visibility, guaranteed conservation of money, and customer account histories. Filmed at qconnewyork.com.

Vitor Olivier is a Software Engineer and Partner at Nubank (Brazil's leading fintech startup). Over the years, he has helped build, maintain, and scale Nubank’s accounting systems, financial products, and securitization strategy.

Publié dans : Technologie
  • Soyez le premier à commenter

Functional / Microservices in Real-Time Financials

  1. 1. Real-time Financials 
 with Microservices and
 Functional Programming Vitor Guarino Olivier
 vitor@nubank.com.br @ura1a
 https://nubank.com.br/
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ nubank-financial-systems
  3. 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. MAIN PRODUCT Live since September 2014
  5. 5. A TECHNOLOGY DRIVEN APPROACH TO FINANCIAL SERVICES
  6. 6. CONTINUOUS DELIVERY
  7. 7. MICROSERVICES
  8. 8. INDEPENDENTLY AND CONTINUOUSLY DEPLOYABLE
  9. 9. DECOUPLED AND EASY TO REPLACE
  10. 10. BOUNDED BY CONTEXT AND INDEPENDENTLY DEVELOPED
  11. 11. WHAT HAPPENS WHEN WE NEED TO COMBINE DATA ACROSS SEVERAL SERVICES? ESPECIALLY IN REAL-TIME
  12. 12. - Running on AWS, 2 AZs, config as code, 
 immutable infra, horizontally scalable,
 sharded by customers SERVICE ARCHITECTURE - Producer/Consumer to Kafka - REST APIs REST - Written in Clojure (functional) - Persistence with Datomic
  13. 13. DATOMIC - Immutable, append-only database - ACID on writes (atomic, consistent, isolated, durable) Lucas Cavalcanti & Edward Wible - Exploring four hidden superpowers of Datomic - A database that works a lot like
  14. 14. The Problem
  15. 15. WE HAVE OVER 90 SERVICES
  16. 16. THE PROBLEM:
 A LOT OF BUSINESS LOGIC DEPENDS ON DATA ACROSS MANY SERVICES Should I authorize a purchase? Should I block a card? Should I charge interest? Purchases InterestChargebacksPayments Currencies
  17. 17. THE PROBLEM:
 WE ARE SHOWING THESE NUMBERS TO THE CUSTOMER IN REAL TIME
  18. 18. THE PROBLEM:
 NO CANONICAL DEFINITION OF OUR KEY NUMBERS - Ad-hoc definitions created by analysts and engineers - Analysis vs. operational definition gap - Nubank, investors, customers, and regulators 
 are all worried about the same numbers.
  19. 19. A BALANCE SHEET IS THE CANONICAL WAY OF REPRESENTING FINANCIAL INFO - We can apply generally accepted accounting principles 
 (verifiable, unbiased) - Conservation of money (every credit should have a debit) - One of the original event-sourced systems LIABILITY ASSET EQUITY
  20. 20. THE MODEL - Entry: represents a debit and a credit to two book-accounts - Meta-entity: it’s a reference to the external entity that originated the event - Movement: a collection of entries. Maps one Kafka message to one db transaction - Book-account: A customer owned balance sheet account
 ex: cash, prepaid, late, payable -Algebraic Models For Accounting Systems 
 by Salvador Cruz Rambaud and José Garcia Pérez - Balance: cumulative sum of entries of a book account
  21. 21. Double-entry accounting
 service
  22. 22. OUR GOAL FOR OUR ACCOUNTING LEDGER
 (aka DOUBLE-ENTRY SERVICE) - High availability to other services, clients, and analysts in real-time - Resilient to distributed systems craziness - Traceability of when and why we were inconsistent (strong audit trail) - Event-driven, via kafka. (we could subscribe to existing topics)
  23. 23. THE IDEAL FLOW f(payload) MOVEMENT[ ] EVENT ACID transaction - Event ordering doesn’t matter - No mutable state - Needs to guarantee all events are consumed - Thread safe
  24. 24. {:purchase
 {:id (uuid)
 :amount 100.0M
 :interchange 1M
 :post-date "2016-12-01"}} Initial Balances: 
 Current Limit R$ 1000, Current Limit Offset R$ 1000 Final Balances: 
 Current Limit: R$ 900, Current Limit: Offset R$ 900
 Settled Purchase: R$ 100, Payable: R$ 99, Interchange Revenue: R$ 1 [{:entry/id (uuid)
 :entry/amount 100.0M
 :entry/debit-account :asset/settled-purchase
 :entry/credit-account :liability/payable
 :entry/post-date "2016-12-01"
 :entry/movement new-purchase} recognize
 receivable/payable {:entry/id (uuid)
 :entry/amount 100M
 :entry/debit-account :liability/current-limit
 :entry/credit-account :asset/current-limit
 :entry/post-date "2016-12-01"
 :entry/movement new-purchase} reduce limit {:entry/id (uuid)
 :entry/amount 1M
 :entry/debit-account :liability/payable
 :entry/credit-account :pnl/interchange-revenue
 :entry/post-date "2016-12-01"
 :entry/movement new-purchase}
 ] recognize
 revenue
  25. 25. WE CAN'T GUARANTEE CONSISTENCY,
 BUT WE CAN MEASURE IT f(payload) MOVEMENT[ ] ACID transaction - Kafka Lag - Service downtime - Processing time produced-at vs. consumed-at post-date vs. produced-at consumed-at vs. db/txInstant
  26. 26. PURE FUNCTIONS OF THE PAYLOAD WON'T ALWAYS WORK
  27. 27. The Stateful Flow
  28. 28. {:payment
 {:id (uuid)
 :amount 150.00M
 :post-date "2016-12-01"}} [{:entry/id (uuid)
 :entry/amount 100.0M
 :entry/debit-account :asset/cash
 :entry/credit-account :asset/late
 :entry/post-date "2016-12-01"
 :entry/movement new-payment} amortize debt {:entry/id (uuid)
 :entry/amount 100M
 :entry/debit-account :asset/current-limit
 :entry/credit-account :liability/current-limit
 :entry/post-date "2016-12-01"
 :entry/movement new-payment} increase limit {:entry/id (uuid)
 :entry/amount 50M
 :entry/debit-account :asset/cash
 :entry/credit-account :liability/prepaid
 :entry/post-date "2016-12-01"
 :entry/movement new-payment}
 ] recognize
 prepaid amount Initial Balances: 
 Current Limit: R$ 900, Current Limit Offset: R$ 900
 Late: R$ 100, Payable: R$ 99, Interchange Revenue: R$ 1 Final Balances: 
 Current Limit: R$ 1000, Current Limit Offset: R$ 1000
 Cash: R$ 150, Prepaid R$ 50, Payable: R$ 99, Interchange Revenue: R$ 1
  29. 29. {:payment
 {:id (uuid)
 :amount 150.00M
 :post-date "2016-12-01"}} [{:entry/id (uuid)
 :entry/amount 100.0M
 :entry/debit-account :asset/cash
 :entry/credit-account :asset/late
 :entry/post-date "2016-12-01"
 :entry/movement new-payment} amortize debt {:entry/id (uuid)
 :entry/amount 100M
 :entry/debit-account :asset/current-limit
 :entry/credit-account :liability/current-limit
 :entry/post-date "2016-12-01"
 :entry/movement new-payment} increase limit {:entry/id (uuid)
 :entry/amount 50M
 :entry/debit-account :asset/cash
 :entry/credit-account :liability/prepaid
 :entry/post-date "2016-12-01"
 :entry/movement new-payment}
 ] recognize
 prepaid amount Initial Balances: 
 Late: R$ 100 Final Balances: 
 Cash: R$ 150, Prepaid R$ 50
  30. 30. THE STATEFUL FLOW - Movements in the past will modify all future balances - Adapters are a function of the event payload AND current balances - Balances can’t change during calculations - Can’t allow for data to be corrupted depending on the order of the events INVARIANTS
  31. 31. INVARIANTS - Some balances can’t coexist (no late alongside prepaid) - We can establish invariants that must hold true at all times - Some balances can’t be negative (cash) - Some can’t be positive (credit-loss)
  32. 32. THE STATEFUL FLOW EVENT f(payload, ) ACID transaction f( ) VALID STATE? FIX VIOLATION INVARIANT VIOLATIONS? NO MOVEMENT[ ] Cr: Late
 Dr: Cash
 R$ 150 Cr: Late
 Dr: Cash
 R$ 150 MOVEMENT WITH CORRECTION[ ] Cr: Prepaid
 Dr: Late
 R$ 50 Initial Balances: 
 Late: R$ 100 Final Balances: 
 Cash: R$ 150, Prepaid R$ 50 [ ] VIOLATIONS YES Negative 
 Late 
 Balance
  33. 33. CHALLENGES
  34. 34. CHALLENGES - Fixing invariants logic is extremely complex. - Datomic indexing is tested until 10 billion facts. - Datomic isn’t the best option for analytical workload, especially with sharded dbs - Other services bugs may generate incorrect entries that will need to be fixed
  35. 35. GENERATIVE TESTING - We generate random events from our schemas (bill, purchases, payments, etc) - Write a function that describes a property that should always hold true 
 instead of describing input and expected output,
 - Properties that should hold true are the same invariants that are guaranteed in prod - Embed the least amount of domain logic assumptions
  36. 36. GENERATIVE TESTING (def balances-property (prop/for-all [account (g/generator Account)
 events (gen/vector (gen/one-of [(g/generator Purchase)
 (g/generator Payment)
 ...]))]
 (->> datomic
 (consume-all! account events)
 :db-after
 (balances-are-positive!))) (fact (tc/quick-check 500 balances-property) => (th/embeds {:result true})) (ns double-entry.controllers.rulebook-test
 (:require [midje.sweet :refer :all] [clojure.test.check.properties :as prop]
 [clojure.test.check :as tc]
 [schema-generators.generators :as g]
 [clojure.test.check.generators :as gen]))
  37. 37. MONITORING / REPLAY HISTORY TOOLING - Other services have republish endpoints 
 (same payload and meta data as original thanks to datomic) - We set sanity checks to make sure events aren’t missing - We have an endpoint that can retract all entries for a customer
 (resets business timeline, but not DB)
  38. 38. SHARDING BY CUSTOMER / TIME - No cross customer entries allows for per customer sharding - We shard the database by time fairly often. - simple representation of the end state of the customer at a time shard: 
 final balance of each of the book accounts - As time passes, any single customer’s db will approach infinite datoms
  39. 39. ETL extract logs facts to table
 (one per entity type) tables stored applies functions to generate balances balances on redshift* * also accessible through metabase
  40. 40. The Result
  41. 41. Text placeholder 2015-01 2015-04 2015-07 2015-10 2015-01 2016-03 REAL TIME BALANCE SHEET
  42. 42. 2 TIMELINES ACTUAL (DB) TIME audit trail / Datomic log “when did we know”day 0 day 30 day 90 BUSINESS TIME official version of events uses business-relevant “post dates” can correct after the fact day 0 day 30
  43. 43. WHAT WE LIKE - Canonical definition of our most important numbers - Financial analysis applied at a the customer level in real-time - Business-specific invariants provide safety - Generative testing finds real bugs - Ability to replay history for a customer without losing data - Shardable by time and by customer - Extensible to other products (some don’t require stateful approach) - Inconsistency traceability allows us to react to it
  44. 44. 42 nubank.com.br/jobs vitor@nubank.com.br @ura1a https://gist.github.com/ura1a to get snippets of our domain! THANK YOU!
  45. 45. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/nubank- financial-systems

×