The way Architecture is done in BigTech, top-down “rules” and technology choices. We will talk about freedom in making architecture decisions for Product and Core teams. We'll go deeper into Fintech solution, built as a crucial component of global SaaS: Microservices, APIs and Queues, Event Sourcing, Feature Toggles, SDLC, CI/CD, DevOps, Monitoring, Analytics and more.

2. Nuclear Industry Online Games
Volodymyr Malyk
Fintech
● 12+ years in HighLoad and BigData
● Python, PHP, Scala, R, C++ and more
● Co-organizer of the International Software
Architects Club committee

3. team of 5,000
● Digital Business and Online Presence
210M users in 190 countries
● SaaS Website Platform (Software as a Service)
Online Stores, Restaurants, Hotels, Travel, Events, and more
● 5% websites, 20+ data centers/PoPs Multi Clouds
8k builds a day, 450 deployments a day, 500M LoC

4. 1. Product & Core, rules and tools
2. Microservices and Request flow
3. Payments at Wix and Lessons Learned
Agenda
Fintech inside of a SaaS powdered by 2000+ Microservices
↓

5. Product & Core
principles and tools
↓

6. Product (75%)
Core & Infra (25%)
FRs
Functional Requirements

7. Product (75%)
Core & Infra (25%)
Technologies
and Open-source
Frameworks, libs, etc
Big data, analytics, etc
CI/CD, Alerts, etc
...
FRs
Functional Requirements

8. Product (75%)
Core & Infra (25%)
Technologies
and Open-source
Business Impact
Online Stores
Restaurants
Hotels
...
Frameworks, libs, etc
Big data, analytics, etc
CI/CD, Alerts, etc
...
FRs
Functional Requirements

9. Product (75%)
Core & Infra (25%)
Business Impact
Velocity & Delivery
NFRs: ab-tests, feature-toggles, ...
Modifiability
NFRs: dependencies, cost-effort, ...
Maintainability
NFRs: lifespan, ...
Performance
NFRs: CPU & Memory, Latency, ...
Availability
NFRs: downtime XX.XX nine’s, ...
Backward compatibility
NFRs: deprecated N days support, ...
...
...
Technologies
and Open-source
QA and NFRs
Quality Attributes and Non-Functional Requirements

10. Product (75%)
Core & Infra (25%)
Business Impact
Velocity & Delivery
NFRs: ab-tests, feature-toggles, ...
Modifiability
NFRs: dependencies, cost-effort, ...
Maintainability
NFRs: lifespan, ...
Performance
NFRs: CPU & Memory, Latency, ...
Availability
NFRs: downtime XX.XX nine’s, ...
Backward compatibility
NFRs: deprecated N days support, ...
...
...
Technologies
and Open-source
QA and NFRs
Quality Attributes and Non-Functional Requirements

11. Principles
1. System Design skill is a MUST
2. API-First: governance, style guide, documentation portal
3. Production-Only: Feature Toggles + AB Tests, TDD
Tools
● 2k+ microservices: CI/CD, Monitoring, API Registry, ...
● 30 min to scaffold & register a new one
● Scala, TypeScript, Node.js, Serverless, gRPC, Kafka and more

12. Principles
1. System Design skill is a MUST
2. API-First: governance, style guide, documentation portal
3. Production-Only: Feature Toggles + AB Tests, TDD
Tools
● 2k+ microservices: CI/CD, Monitoring, API Registry, ...
● 30 min to scaffold & register a new one
● Scala, TypeScript, Node.js, Serverless, gRPC, Kafka and more

13. Tools
● 2k+ microservices: CI/CD, Monitoring, API Registry, ...
● 30 min to scaffold & register a new one
● Scala, TypeScript, Node.js, Serverless, gRPC, Kafka and more
Principles
1. System Design skill is a MUST
2. API-First: 101 guidelines, documentation portal, governance, tools
3. Production-Only: Feature Toggles + AB Tests, TDD

14. Principles
1. System Design skill is a MUST
2. API-First: 101 guidelines, documentation portal, governance, tools
3. Production-Only: Feature Toggles + AB Tests, TDD
Tools
● 2k+ microservices: CI/CD, Monitoring, API Registry, ...
● 30 min to scaffold & register a new one
● Scala, TypeScript, Node.js, Serverless, gRPC, Kafka and more

15. Tools
● 2k+ microservices: CI/CD, Monitoring, API Registry ...
● 30 min to scaffold & register a new one
● gRPC, Kafka, Scala, TypeScript, Node.js, Serverless and more
Principles
1. System Design skill is a MUST
2. API-First: 101 guidelines, documentation portal, governance, tools
3. Production-Only: Feature Toggles + AB Tests, TDD

16. Microservices and Request Flow
↓

17. Microservices and Request flow
Visitors Load Balancing
(incl. TLS termination)

18. Microservices and Request flow
Visitors API Gateway
- Http Mapping
- Enrich: Sessions, Permissions,
Feature-Toggles, Time budget,
Tracing, etc
Load Balancing
(incl. TLS termination)

19. Microservices and Request flow
Visitors API Gateway
- Http Mapping
- Enrich: Sessions, Permissions,
Feature-Toggles, Time budget,
Tracing, etc
Mesh
Mesh
Mesh
Dispatchers
Ingress & Egress
Load Balancing
(incl. TLS termination)

20. Microservices and Request flow
Visitors API Gateway
- Http Mapping
- Enrich: Sessions, Permissions,
Feature-Toggles, Time budget,
Tracing, etc
Mesh
Mesh
Mesh
Dispatchers
Ingress & Egress
Load Balancing
(incl. TLS termination)
Microservices
2k+
FooMsvc
BarMsvc
2kGetMsvc
...

21. Microservices and Request flow
Visitors API Gateway
- Http Mapping
- Enrich: Sessions, Permissions,
Feature-Toggles, Time budget,
Tracing, etc
Mesh
Mesh
Mesh
Dispatchers
Ingress & Egress
Load Balancing
(incl. TLS termination)
Microservices
2k+
FooMsvc
BarMsvc
2kGetMsvc
...
Pods
20k+

22. Microservices and Request flow
Visitors API Gateway
- Http Mapping
- Enrich: Sessions, Permissions,
Feature-Toggles, Time budget,
Tracing, etc
Mesh
Mesh
Mesh
Dispatchers
Ingress & Egress
Load Balancing
(incl. TLS termination)
Microservices
2k+
FooMsvc
BarMsvc
2kGetMsvc
...
Pods
20k+

23. Payments at Wix
and Lessons Learned
↓

24. Payments at Wix
Visitor
User of User
Merchant
User

25. Payments at Wix
Visitor
User of User
Shop
Food
Travel
Merchant
User

26. Payments at Wix
Checkout
Visitor
User of User
Shop
Food
Travel
Merchant
User

27. Payments at Wix
Checkout
Visitor
User of User
Shop
Food
Travel
Merchant
User

28. Architecture
Website
Shop
Hotel
Restaurant
...
Merchant

29. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Merchant
...

30. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Merchant
...
Sync calls
Webhooks

31. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
...
Sync calls
Webhooks

32. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
...
Sync calls
Webhooks

33. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
...
Sync calls
Webhooks

34. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
...
Sync calls
Webhooks

35. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
...
Sync calls
Webhooks

36. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
...
Sync calls
Webhooks

37. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
...
Sync calls
Webhooks

38. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
...
Sync calls
Webhooks
a lot
of Payment Providers
Worldwide

39. Payment Providers Worldwide
● Quality Attribute: Velocity & Delivery
● Solution: transform NFRs to FRs

40. Payment Providers Worldwide
● Quality Attribute: Velocity & Delivery
● Solution: transform NFRs to FRs

41. Payment Providers Worldwide
● Quality Attribute: Velocity & Delivery
● Solution: transform NFRs to FRs
Integrate via API — NO
Expose SPI — YES!

42. Service Provided Interface (SPI)
● An API intended to be implemented by a third party.
● Yes, that’s like a plugin system.
● The same as if all Payment Providers had the same API for us:
○ Connect Account
○ Create Transaction
○ Refund Transaction
○ Webhooks on payment state
○ Idempotency

43. Service Provided Interface (SPI)
● API-First — new Revenue Stream!
● Documentation as a Product
● Still have uncovered cases

44. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
Sync calls
Webhooks
a lot
of Payment Providers
Worldwide :)

45. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
State?
Traceability?
Sync calls
Webhooks

46. Event-Sourcing
Business-logic Store
State
(Model,
ORM, etc)

47. Event-Sourcing
Business-logic Store
Domain events
State
(Aggregate)
Business-logic Store
State
(Model,
ORM, etc)

48. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Store
Business-logic Store
State
(Model,
ORM, etc)

49. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Store
Business-logic Store
State
(Model,
ORM, etc)

50. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Store

51. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store

52. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc

53. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc
Supportive Side-Effects
email, BI, etc

54. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc
Supportive Side-Effects
email, BI, etc
● MySQL as a storage
● Events are stored by tenant id + type
● Events are backward compatible
● Rebuild Aggregate in-memory per request
● No snapshots
● Events migration is OK

55. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc
Supportive Side-Effects
email, BI, etc
● MySQL as a storage
● Events are stored by tenant id + type
● Events are backward compatible
● Rebuild Aggregate in-memory per request
● No snapshots
● Events migration is OK

56. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc
Supportive Side-Effects
email, BI, etc
● MySQL as a storage
● Events are stored by tenant id + type
● Events are backward compatible
● Rebuild Aggregate in-memory per request, per tenant
● No snapshots
● Events migration is OK

57. Event-Sourcing
Business-logic Domain events
State
(Aggregate)
State under Feature Toggle
(custom Aggregate)
Analytics and BI
Commands Store
Crucial Side-Effects
payments, etc
Supportive Side-Effects
email, BI, etc
● MySQL as a storage
● Events are stored by tenant id + type
● Events are backward compatible
● Rebuild Aggregate in-memory per request, per tenant
● No snapshots
● Events migration is fine

58. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
Event Sourcing
Sync calls
Webhooks

59. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
Event Sourcing
Sync calls
Webhooks
Traceability?

60. ● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper - API to encapsulate DB interaction
● Mask PII
Logs?

61. ● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper - API to encapsulate DB interaction
● Mask PII
Logs?

62. ● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper - API to encapsulate DB interaction
● Mask PII
Logs?
PII GDPR

63. ● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper - API to encapsulate DB interaction
● Mask PII
Logs?
PCI DSS
PII GDPR

64. ● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper - API to encapsulate DB interaction
● Mask PII
Logs?
PCI DSS
Confidential
PII GDPR

65. Audit Logs
● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
● Wrapper — API to encapsulate DB interaction
● Mask PII: full names, email addresses, etc

66. Audit Logs
● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
○ Wrapper — API to encapsulate DB interaction
● Mask PII: full names, email addresses, etc
● No Credit Card data!
Confidential

67. Audit Logs
● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
○ Wrapper — API to encapsulate DB interaction
● Mask PII: full names, email addresses, etc
● No Credit Card data!
Confidential
PII GDPR

68. Audit Logs
● Store requests to 3rd parties
● Store responses from 3rd parties
● Store Webhooks
● Encrypted “logs” in DB, by business entities ids
○ Wrapper — API to encapsulate DB interaction
● Mask PII: full names, email addresses, etc
● No Credit Cards data!
PCI DSS
Confidential
PII GDPR

69. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
Event Sourcing
Audits
Sync calls
Webhooks

70. Architecture
Website
Shop
Hotel
Restaurant
...
PayPal
Stripe
Integrations
Account Management
- encrypted api keys
- p-methods setup
- status check
Metadata
- Providers & Methods
- Worldwide adjust
Merchant
Checkout
Processing
- transactions
- persisted stateful
Dashboard
SPI
...
Event Sourcing
Audits
Sync calls
Webhooks

71. Where are your Architects?
↓

72. ● Engineers
● System Design skill is a MUST
Where are your Architects?
↓
↓

73. ➔ GitHub: https://github.com/wix
➔ Telegram: https://t.me/wixeng
➔ Youtube:
https://www.youtube.com/c/WixTechTalks
➔ Medium: https://medium.com/wix-engineering
Thank You!