Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Modern Apps: Enriching the
User Experience with HTAP
Thomas Boyd
Director, Technical Marketing
MariaDB Corporation
WHAT WE’RE FOCUSING ON
● User Experience and Data Access
● Use Cases
● MariaDB Platform & HTAP
○ MariaDB Server
○ MariaDB ...
User Experience &
Data Access
Speed
Flexible Data Exploration
Access to Historical and Current Data
● Realtime
● Simultaneous
● Summary and Drilldown
Use Cases
Retail: Personalized Shopping Experience
● Amplify sales by providing a
personalized shopping experience
across all the ch...
In the real world
● Institute for Health Metrics and Evaluation
○ Global population health analysis
○ 30TB of data
○ 100 b...
MariaDB Platform &
HTAP
MariaDB Platform
● Replication and clustering
● Sharding
● Compression
● Encryption
● Auditing
● Point-in-time rollback
● ...
"In Flight" Transaction Analytics
"In Flight" analytics"After the fact" analytics
MariaDB Server
MariaDB Server
● Replication and clustering
● Sharding
● Compression
● Encryption
● Auditing
● Point-in-time rollback
● Ve...
MariaDB Maxscale
MariaDB MaxScale
● Automatic failover
● Transaction replay
● Schema sharding
● Casual reads
● Query blocking
○ Database fi...
MariaDB MaxScale
MariaDB MaxScale: database proxy
Session
throttle
Query result
cache
Query
router
Database
firewall
Respo...
Database
Database
MariaDB MaxScale
Server
(server -> IP)
Server
(server -> IP)
Listener
(port -> service)
Service
(request...
OLTP server 1
(transactional)
[sw-db1]
type = server
address = <MariaDB-Server-1-IP>
port = 3306
protocol = MariaDBBackend...
OLAP server 1
(analytical)
[sw-mcs-um2]
type = server
address = <MariaDB-UM-1-IP>
port = 3306
protocol = MariaDBBackend
OL...
OLAP listenerOLTP listener
[MDB-Listener]
type = listener
service = MDB-Service
protocol = MariaDBClient
port = 4009
[CS-L...
HTAP service
[HTAP-Service]
type = service
router = schemarouter
ignore_databases_regex = .*
servers = MDB-Service-as-serv...
Database instance
(transactional)
Database instance
(transactional)
Database instance
(analytical)
Database instance
(tran...
Database instance
(analytical)
Database instance
(transactional)
Database instance
(transactional)
Database instance
(tran...
Database instance
(transactional)
Database instance
(transactional)
Database instance
(analytical)
Database instance
(tran...
MariaDB
ColumnStore
ColumnStore overview
● Distributed and columnar storage for analytics
○ Stores data by column rather than row (file per co...
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
col1 col2 … col50
1 2 10 100
4 12 50
3 8 6 75
4 6 18 200
2
Performance Module (PM)
User Module (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
Rows 1 to 300,000
90ms
S3 (PM) S4 (PM)S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
ColumnStore
Storage
ColumnStore
Stora...
S3 (PM) S4 (PM)S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
ColumnStore
Storage
ColumnStore
Stora...
S3 (PM) S4 (PM)S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
ColumnStore
Storage
ColumnStore
Stora...
S5 (PM) S6 (PM)S4 (PM)
S2 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
ColumnStore
Storage
S1 (UM)
MariaDB S...
PMs
Bulk adapters
UMs
MariaDB Server
InnoDB
ColumnStore
ColumnStore
C
Java
Python
Stream adapters
Kafka
MaxScale CDCSpark
...
Q & A
Thank You
Data Con LA 2019 - Hybrid Transactional Analytical Processing (HTAP) with MariaDB Platform by Todd Stoffel
Data Con LA 2019 - Hybrid Transactional Analytical Processing (HTAP) with MariaDB Platform by Todd Stoffel
Data Con LA 2019 - Hybrid Transactional Analytical Processing (HTAP) with MariaDB Platform by Todd Stoffel
Prochain SlideShare
Chargement dans…5
×

Data Con LA 2019 - Hybrid Transactional Analytical Processing (HTAP) with MariaDB Platform by Todd Stoffel

305 vues

Publié le

Learn how companies are using MariaDB Platform to bring Hybrid Transactional Analytical Processing to their Modern Applications

Publié dans : Technologie
  • Login to see the comments

Data Con LA 2019 - Hybrid Transactional Analytical Processing (HTAP) with MariaDB Platform by Todd Stoffel

  1. 1. Modern Apps: Enriching the User Experience with HTAP Thomas Boyd Director, Technical Marketing MariaDB Corporation
  2. 2. WHAT WE’RE FOCUSING ON ● User Experience and Data Access ● Use Cases ● MariaDB Platform & HTAP ○ MariaDB Server ○ MariaDB Maxscale ○ MariaDB ColumnStore
  3. 3. User Experience & Data Access
  4. 4. Speed
  5. 5. Flexible Data Exploration
  6. 6. Access to Historical and Current Data ● Realtime ● Simultaneous ● Summary and Drilldown
  7. 7. Use Cases
  8. 8. Retail: Personalized Shopping Experience ● Amplify sales by providing a personalized shopping experience across all the channels ● OLTP query ○ process purchase transaction ● Analytics ○ tailored promotion based on part and current purchase Personalized Shopping Experience
  9. 9. In the real world ● Institute for Health Metrics and Evaluation ○ Global population health analysis ○ 30TB of data ○ 100 billion rows ● OTC Markets Group ○ Historical trade analysis ○ 10TB of rolling data (5 years) ○ 10,000 U.S. and global securities ● Pinger, Inc. ○ Customer behavioral analysis ○ 30 million texts, 3 million phone calls and 1.5 billion logs per day
  10. 10. MariaDB Platform & HTAP
  11. 11. MariaDB Platform ● Replication and clustering ● Sharding ● Compression ● Encryption ● Auditing ● Point-in-time rollback ● Versioned tables ● CTEs and window functions ● JSON and GIS functions ● Oracle compatibility MariaDB Server ● Enterprise high availability ○ Automatic failover ○ Transaction replay ● Enterprise scalability ○ Read-write splitting ○ Replication server ● Enterprise security ○ Data masking ○ Query blocking ○ Result limiting ● Enterprise integration ○ Change-data-capture ○ Kafka connector MariaDB MaxScale (Platform services) ● Columnar storage ● Distributed data ● Massively parallel processing ● Regression functions ● Spark connector ● Kafka connector ● Pentaho Kettle connector MariaDB ColumnStore
  12. 12. "In Flight" Transaction Analytics "In Flight" analytics"After the fact" analytics
  13. 13. MariaDB Server
  14. 14. MariaDB Server ● Replication and clustering ● Sharding ● Compression ● Encryption ● Auditing ● Point-in-time rollback ● Versioned tables ● CTEs and window functions ● JSON and GIS functions ● Oracle compatibility
  15. 15. MariaDB Maxscale
  16. 16. MariaDB MaxScale ● Automatic failover ● Transaction replay ● Schema sharding ● Casual reads ● Query blocking ○ Database firewall ● Data masking ● Denial-of-service protection ○ Result limiting Database proxy ● Offload replication from the primary database ● Supports a very large number of read replicas Replication server ● Replicates data from transactional database instances to analytical database instances ● Sends write events to external systems ○ Apache Kafka producer ○ CDC Client Change-data-capture server
  17. 17. MariaDB MaxScale MariaDB MaxScale: database proxy Session throttle Query result cache Query router Database firewall Response Request Result limiting Data masking Database monitor MariaDB Enterprise Server Connector Transaction monitor Query logger
  18. 18. Database Database MariaDB MaxScale Server (server -> IP) Server (server -> IP) Listener (port -> service) Service (request -> server) Server (server -> IP) Filter(s)Protocol Monitor Applications Database
  19. 19. OLTP server 1 (transactional) [sw-db1] type = server address = <MariaDB-Server-1-IP> port = 3306 protocol = MariaDBBackend OLTP server 2 (transactional) [sw-db2] type = server address = <MariaDB-Server-2-IP> port = 3306 protocol = MariaDBBackend OLTP server 3 (transactional) [sw-db3] type = server address = <MariaDB-Server-3-IP> port = 3306 protocol = MariaDBBackend OLTP service (transactional) [MDB-Service] type = service router = readwritesplit servers = sw-db1,sw-db2, sw-db3 user = maxuser password = maxpwd OLTP listener (transactional) [MDB-Listener] type = listener service = MDB-Service protocol = MariaDBClient port = 4009
  20. 20. OLAP server 1 (analytical) [sw-mcs-um2] type = server address = <MariaDB-UM-1-IP> port = 3306 protocol = MariaDBBackend OLAP server 2 (analytical) [sw-mcs-um2] type = server address = <MariaDB-UM-2-IP> port = 3306 protocol = MariaDBBackend OLAP service (analytical) [CS-Service] type = service router = readconnroute router_options = running servers = sw-mcs-um1, sw-mcs-um2 user = maxuser password = maxpwd OLAP listener (analytical) [CS-Listener] type = listener service = CS-Service protocol = MariaDBClient port = 4010
  21. 21. OLAP listenerOLTP listener [MDB-Listener] type = listener service = MDB-Service protocol = MariaDBClient port = 4009 [CS-Listener] type = listener service = CS-Service protocol = MariaDBClient port = 4010 OLTP server (virtual) OLAP server (virtual) [CS-Service-as-server] type = server address = 127.0.0.1 port = 4010 protocol = MariaDBBackend [MDB-Service-as-server] type = server address = 127.0.0.1 port = 4009 protocol = MariaDBBackend HTAP service HTAP listener [HTAP-Service] type = service router = schemarouter ignore_databases_regex = .* servers = MDB-Service-as-server,CS-Service-as-server Preferred_server = MDB-Service-as-server user = maxuser password = maxpwd filters = target-selector [HTAP-Listener] type = listener service = HTAP-Service protocol = MariaDBClient port = 4011
  22. 22. HTAP service [HTAP-Service] type = service router = schemarouter ignore_databases_regex = .* servers = MDB-Service-as-server,CS-Service-as-server Preferred_server = MDB-Service-as-server user = maxuser password = maxpwd filters = target-selector [target-selector] type = filter module = namedserverfilter match01 = (?i)SELECT.*loan target01 = CS-Service-as-server match02 = .* target02 = MDB-Service-as-server Filter(s) Regex filter (by query syntax) Schema router (by table/view) Named server filter (by query syntax) Hint filter Options
  23. 23. Database instance (transactional) Database instance (transactional) Database instance (analytical) Database instance (transactional) Database instance (analytical) select avg(total) from tbl_purchases where customer=1 select avg(total) from tbl_purchases --maxscale route to analytics MariaDB MaxScale Query routing: hint filter
  24. 24. Database instance (analytical) Database instance (transactional) Database instance (transactional) Database instance (transactional) Database instance (analytical) select avg(total) from tbl_purchases where customer=1 select avg(total) from view_purchases_history MariaDB MaxScale Query routing: schema router tbl_purchases view_purchase_history tbl_purchases
  25. 25. Database instance (transactional) Database instance (transactional) Database instance (analytical) Database instance (transactional) Database instance (analytical) select avg(total) from tbl_purchases where customer=1 select avg(total) from tbl_purchases_history MariaDB MaxScale Query routing: combo 1. Named server filter (attach hint) select avg(total) from tbl_purchases 2. Regex filter
  26. 26. MariaDB ColumnStore
  27. 27. ColumnStore overview ● Distributed and columnar storage for analytics ○ Stores data by column rather than row (file per column) ○ Stores data on separate, dedicated storage nodes ○ Scale out by adding more storage nodes ○ Can use GlusterFS for high availability ● Compression: up to 90%
  28. 28. MariaDB Server InnoDB ColumnStore ColumnStore Storage col1 col2 … col50 1 2 10 100 4 12 50 3 8 6 75 4 6 18 200 2
  29. 29. Performance Module (PM) User Module (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage
  30. 30. S2 (PM) S1 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage Rows 1 to 300,000 90ms
  31. 31. S3 (PM) S4 (PM)S2 (PM) S1 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage ColumnStore Storage ColumnStore Storage Rows 1 to 100,000 Rows 100,001 to 200,000 Rows 200,001 to 300,000 30ms 30ms 30ms
  32. 32. S3 (PM) S4 (PM)S2 (PM) S1 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage ColumnStore Storage ColumnStore Storage Rows 1 to 100,000 Rows 100,001 to 200,000 Rows 200,001 to 300,000 50 60 70 AVG(50, 60, 70)
  33. 33. S3 (PM) S4 (PM)S2 (PM) S1 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage ColumnStore Storage ColumnStore Storage T1: 00,001 to 30,000 T2: 30,001 to 60,000 T4: 100,001 to 130,000 T5: 130,001 to 160,000 T7: 200,001 to 230,000 T8: 230,001 to 260,000 6040 50 7050 60 8060 70 AVG(40, 50, 60, 50, 60, 70, 60, 70, 80)
  34. 34. S5 (PM) S6 (PM)S4 (PM) S2 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage ColumnStore Storage S1 (UM) MariaDB Server InnoDB ColumnStore S3 (UM) MariaDB Server InnoDB ColumnStore ColumnStore Storage
  35. 35. PMs Bulk adapters UMs MariaDB Server InnoDB ColumnStore ColumnStore C Java Python Stream adapters Kafka MaxScale CDCSpark Storage SQL Import (CSV)
  36. 36. Q & A
  37. 37. Thank You

×