In the context of a fast-paced competitive and regulated KYC market, Quicksign needed to rethink its platform. Our goal: quickly adjust to our customers' workflows and integration requirements, scale to support the growing usage and features of our services, be secured and resilient by design—while being fun to build on!
Using Kafka Streams, the Camunda BPMN engine and some in-house developments, we built on Kafka and Kubernetes a fully reactive and streamed CQRS-ES microservices platform. It exposes, for each of our tenants’ processes and without involving any specific development, a bespoke and reactive REST API.
We’ll share how we leveraged Kafka as the backbone of our architecture to fuel our microservices and Kafka Streams to implement the CQRS-ES architectural pattern which we believe is key to provide the strongest foundation for growth in the coming years.
Rethinking Quicksign's Digital Onboarding - Confluent Streaming Days Paris
1. 1
PARIS - 11 OCTOBRE 2018
Cédric Vidal
CTO
Rethinking digital onboarding: how
QuickSign's new microservices platform
leverages Kafka to massively scale
@cedricvidal
2. Kafka is the backbone of
microservices architecture
4. ~500 million tweets per day
~1,3 million trace entries per day
We are not twitter ...
5. Who are we ?
● QuickSign is the European leader in digital
onboarding for financial services
● White label
● Now handling millions of digital subscriptions
per year
10 years experience
15 countries
+70% Transactions growth / year
6. The state of digital
subscription
● critical channel
● banks, insurances
● major challenge
● performance of the subscription
7. ● fast access to information
● fast decision
● automate the decision process
● push the data to the consumer
Realtime is key to the
decision process
realtime streamed
architectures
8. Artificial intelligence
● game changer
● the need for realtime dialog
● need to process a lot of data
● the machine can have doubts
● ask the human for confirmation
12. CQRS-ES with Kafka
user
Command Query Responsibility Segregation (Event Sourcing)
Aggregation Root
Store
direct modification
commands & queries
Controller
action
evaluates
Command
Event
EventEvent
Event Store
stores
produces
Command
Handler
reads
Event
EventEvent
Aggregation
Root
Event
Handler
13. Why Kafka ?
General observation
• Works for extremely massive big data use
cases
• Should handle easily our CQRS ES based
workloads …
Technical details of interest for CQRS-ES
• At least once guaranteed delivery
• Sequential processing per topic partition
• Sharding by design
• RPC like low latency
• Durability
• Replayability
14. Kafka Streams - The key
Technical details of interest for CQRS-ES
• Automatic offset tracking (inherited from Consumer
API)
• Embedded ephemeral disk based KV Store to persist
intermediate stream states and query stores
(Facebook’s RocksDB)
• Rich streaming DSL with operators (aggregates, joins,
…)
• Interactive Queries
Natural fit for Kubernetes’ stateless model
• a simple static void main which joins the Kafka Stream
cluster
• Very natural with the stateless Kubernetes deployment
model
15. All different
• Specific to each customer/product/country
• Business requirements
• Integration requirements
• Different maturity levels, business wise and technical wise