This document discusses the evolution of Parimatch's sports betting product and its new platform. It covers: 1. Parimatch has evolved its product every 3 years, with yearly betting amount increases of 30%. Its active users have grown from 2,000 to 200,000 from 2004 to 2018. 2. The new BetLab platform was created in 2014 and uses technologies like .NET, Node.js, React, PostgreSQL and Redis. It currently has over 250 employees developing products like a sportsbook, core system and payments hub. 3. The presentation outlines solutions for handling the high loads on sports betting sites around major events, including using Aerospike for scalable transactions and Kafka for real-

1. SPORTS BETTING
EVOLUTION
Constantine Obraztsov
BETLAB

2. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
2
1. Product evolution
2. New Platform
3. Solutions
Agenda

3. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
3
1. Product Evolution

4. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
4
Parimatch - Timeline
2003 2007 2013
YEARS 2003 - 2005
• Starting new era of IT in
Parimatch. Introduced new
solution for Retail stream –
BetManger
• One year after, release new
website and started in house
development of Casino product.
• 2005 new stream – phone betting
(using TeleText technology)
YEARS 2007 – 2013
• Mobile application for Symbian
OS (4 versions)
• 2010 first launch of new brand
(Belorussia)
• 2010 main web site were
completely re-designed
YEARS 2013 – 2017
• 2016 Mobile website on new
Platfrom
• 2017 new Desktop website
launch (new technologies)
• 2017 Native application launch
(iOS / Android)

5. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
5
Product evolution - Facts
Each 3 years product line
getting new design
Yearly increase bets
amount to 30%
2nd place in the world
according to similarweb

6. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
6
Flashback 2004 - 2018
10 000
2004 2018
Active Users 2000 200 000
Bets per day 10000 1 000 000 +
Events amount 500 10 000
Hardware 3 servers 200 + servers
IT team 3 employee 250 + employee

7. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
7
• PHP – legacy FE solution
• Delphi - backend legacy solution
• MSSQL – storage legacy solution
Bet Manager:

8. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
8
3. New Platform

9. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
9
BETLAB - Start
— Born: 2014
— Goal: build Sports betting platform
— Targets: stable, scalable, adaptive
9

10. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
→ 4+years
→ 250+ employees
→ ¾ of our Technical staff are Senior
professionals
→ Each person from our product
team has mature betting
experience
BETLAB IN FIGURES:
For the last 4 years we have grown to
200+ employees and continue to grow
hiring 10 new employees every month.
10

11. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
11
• .NET, .NET Core - backend for new developments
• Node.js - real time updates (feed)
• Riot.js->React , Vue.js - front-end frameworks
• Redis - cache, pub/sub
• Kafka, RabbitMQ - reliable messaging
• PostgreSQL, Mongodb, Elastic, Aerospike - storage
BETLAB – Technologies

12. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
12
• Frontend web solution
• Native applications
• DWH
• Sportsbook
• Core
• Payments HUB
Our products

13. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Current production

14. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
New Platform (November)

15. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Third-party solutions
15
Development environment
MicrosoftVisual Studio
Intellij WebStorm
Delphi
MicrosoftVisual Studio for Mac
MicrosoftVisual Studio Code
Atom
Sublime
Microsoft SQL Management Studio
SCM, CI, CD
Gitlab and Gitlab CI
TeamCity
Jenkins
Octopus
JIRA
Confluence
Storage
PostgreSQL
Microsoft SQL Server
mongodb
Aerospike
Cache
Redis
Memcached
Messaging
Kafka
RabbitMQ
Redis
ZMQ
Cluster management
Zookeeper
Discovery, hosting, rollout
HAProxy
Cloudflare
Nomad
Consul
Docker
Kubernetes
etcd
Monitoring
Zabbix
Prometheus
Newrelic
Grafana
Log processing
Elastic stack (Filebeat, Logstash, Elastic
search, Kibana)

16. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
16
Highload - sports betting

17. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Highload - load on site before match
17
• Active Users : 25 000
• Passive users: ±5 000
• Placebet: 3500 (min)
• Transactions: 5000 (min)
• Logins: 500 (min)

18. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Before match

19. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Highload - kick off
19

20. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
After match

21. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
HighLoad - finish
21

22. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
22
2. Solutions

23. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Monolith architecture - Wallet
• BD server - 96 CPUs with 1TB of RAM
• Bets Settlement - 1 000 000 in 5 minutes
• Bets Placement - 4 000 per second
• Seamless wallet – not possible
AEROSPIKE
• Separate component - the only task is process translations fast
• Financial transactions - up to 300 000 per second (per node)
• Horizontally scalable on service and storage level

24. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Communication solution
• Live odds - changes each second (like in finance)
• Average update flow - 30Mbit per sec from BD
• Pull mode - 5-6 sec to pull full set of odds
• Outdated odds – on highload timeframes about 20%
KAFKA
• Communication HUB - feed/bets/odds/settlement
• Real time odds - updated odds is less then a second (outdated ±2%)
• Independent components - work model

25. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
KUBERNETES as scaling solution
• Could ready - took week time to run-up in Amazon our new Platform
• Dynamic scaling - on highload we increasing wallet instances dynamically
• Unification - single configuration and customization place
• Hardware resource optimizations – on demand resources scaling

26. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
26
Conclusion
KEY
COMPONENTS
• New Sportsbook
• New Core
• New Feed
PM International (USD/EUR)
• Main brand (.com)
• Only new users
• Catch up on current
product features
• Migration of Cyprus
region
NOVEMBER
2018
MAY
2019
PM International full
migration
KEY
COMPONENTS
• New Sportsbook
• New Core
• New Feed
• Catch up on current
product features
• Migration of Cyprus
region

27. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Conclusion
New platform kick-off
(Brazil, Turkey, etc.)
Migration of Parimatch Russia brand Final milestone
Full
migration
of brands
AUGUST
2019
JUNE
2020
Full migration
of brands

28. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Conclusion
PM International (USD/EUR)
PM International full
migration
Migration of PM
Russian brand
New Platforms
(brands) set up
Full migration
from current
Platform
NOVEMBER
2019
MAY
2019
AUGUST
2019
JUNE
2019

29. BetLab|Highloadfwdays'1815.09|KonstantinObraztsov
Conclusion
Thank you for attention
kostya@betlab.com