Presented at FOSSASiA19

1. POLARDB:
A database architecture for
the cloud

2. ØYSTEIN GRØVLEN
Sr. Staff Engineer @ Alibaba Cloud
Bio:
Before joining Alibaba, Øystein worked for 10 years in the
MySQL optimizer team at Sun/Oracle. At Sun Microsystems,
he was also a contributor on the Apache Derby project and
Sun's Architectural Lead on Java DB. Prior to that, he worked
for 10 years on development of Clustra, a highly available
DBMS.

3. POLARDB：a Cloud Native Database
Emerging
Hardware
• NVM
• RDMA
• FPGA
Serverless
• Auto Scaling
• Paid by Usage
• Zero Downtime
Security
• Encryption
• Audit
• Access Control
Intelligence
• Self-configuration
• Self-optimization
• Self-diagnosis
• Self-healing
CLOUD NATIVE
User Oriented

4. Database Architecture Revolution : Separation of Storage and Computation
Transaction
Architecture: Separation of Storage and Computation
Database Storage Engine
Computation OffloadingStorage
Compatibility
SecurityHTAPMulti-Model
Usability
Self-Driving
Manageability

5. Cloud Native Architecture
• Scale compute and storage independently
• Shared storage
• Across AZ fail-over without data loss
• Optimize division of functionality between
storage and compute
• Tight integration with other cloud components
like metering, monitoring, control plan
• Optimize for hardware in the data centers
• Compatible with MySQL/PG etc
• Security
PolarProxy
PolarStore
POLARDB
Intelligent proxy
100% Compatible
Storage Optimized
For Database
PolarFS

6. PolarStore: Architecture overview
- Design for Emerging Hardware
- Low Latency Oriented
- Active R/W – Active RO
- High Availability
libpfs
Host1
POLARDB
libpfs
POLARDB
Host2
volume 1 Volume 2
chunk1 chunk2 chunk1 chunk2
PolarSwitch
libpfs
POLARDB
volume 1
PolarSwitch
chunk1 chunk2
ChunkServer ChunkServer ChunkServer ChunkServer
chunk chunk chunk chunk
ParallelRaft
PolarCtrl
metadata
Key Components: 1. libpfs 2. PolarSwitch 3. ChunksServer 4. PolarCtrl
data route
control route

7. PolarStore: Design for Emerging Hardware
- No Context Switch
- OS-bypass & zero-copy
RDMA-NIC
Network Over RDMA
libpfs
POLARDB
Memory
- Parallel Random I/O absorbed by Optane
- Excellent performance with less long tail latency issue
- No need of Over Provisioning
WAL Log in 3Dxpoint optane
RDMA Network
RDMA
RDMA-NIC
Optane
NVMe SSDs
Memory
Chunkserver 1
RDMA-NIC
Optane
NVMe SSDs
Memory
Chunkserver 3
RDMA-NIC
Optane
NVMe SSDs
Memory
Chunkserver 2
PolarDB write to shm

8. PolarFS: posix distributed file system closely with DB
Pure User Space
For Extra-low Latency
- No Sys call
- No Context Switch
- Zero Data Copy
Posix Semantics
- Easy Porting
Node 1
libpfs
POLARDB
Journal file
Paxos file
Low Latency Oriented
libpfs
POLARDB
libpfs
POLARDB
Node 2 Node 3
1 2 3
4
5 6
head
pending tail
tail
POLARDB Cluster File System Metadata Cache
Directory Tree File Mapping Table
root FileBlk VolBlk
0
1
2
…
348
1500
0 201
…
6
Database Volume
Chunks
…
Block Mapping Table
FileID FileBlk
489
478
…
16
0 201
…
VolBlk
200
201
202
0 2010 316
…
3
PolarFS: An Ultra-low Latency and Failure Resilient Distributed File System for Shared
Storage Cloud Database （VLDB 2018）

9. Dynamic Scaling
Local
Storage
Fast Scaling
MySQL
POLARDB
Master
Local
Storage
Replica
Local
Storage
Replica
Master Replica Replica
Shared Storage
Upgrade 2vCPU to 32vCPU， only in 5 minutes
Add more Replicas, only in 5 minutes.
数值轴
1 Replica 2 Replica 3 Replica 4 Replica 5 Replica 10 Replica
20,949
11,349
9,749
8,149
6,549
4,949
39,844
20,102
16,811
13,521
10,230
6,940
RDS MySQL POLARDB
Lower Cost: 30%~50% OFF
Total costs of 4vCPU 32G Memory 500G Storage with
different replica numbers
0
10000
20000
30000
40000

10. Shared Nothing Logical Replication vs. Shared Storage Physical Replication
Local Storage Local Storage
Master
POLARSTORE
Slave Master Slave
Data
Binlog
Redo
log Data
Master
Binlog
Slave
Binlog
Redo
log
Data
Redo
log
Data
Redo
log
Binlog
Physical Replication is much more reliable than Logical Replication

11. Shared Nothing Logical Replication vs. Shared Storage Physical Replication
Non-blocking low-latency DDL synchronization
Master
Slave
Timeline
Add Column
Running 1 Hour
Add Column
Blocked 1 Hour
Applying DDL will block following events
Add Column
Update
data files
Update metadata
Need not modify data files
MySQL POLARDB
Shared Storage
Master
Slave

12. Physical Replication by Redo Log
Commit
Async Flush
Data File Redo Log
DATA LOG & MEMORY
Primary
Shared Storage
Log Parse
Hash
Table
Redo
Buffer
Pool
Buffer Pool
Write Memory
Query
Snapshot of T4
T2
T4
T5
T1
T3
T3T2T1 T4 T5
T3T2T1 T4
T3T2T1 T4 T5
RO Node
T4
Transactions
Buffer Pool
Shared Storage Continuous Recovery Consistent Snapshot Read
T1

13. Physical Replication - Page from Past
Oldest read view
Control purge
Avoid Data Gap
Checkpoint LSN
(T1)
Primary
Shared Storage
Log Parse
Hash
Table
Redo
Buffer Pool
Snapshot of T4
T2
T4
T5
T1
T3
T3T2T1 T4 T5
T3T2T1 T4
T1
T4Buffer Pool
Data
Redo Log
Checkpoint
T1
T3T2T1 T4
Purgeable Unpurgeable
RO
Node
Primary
RO Node

14. Physical Replication - Page from Future
Avoid Data Overstep
Control flush datafile
Primary
Shared Storage
Log Parse
Hash
Table
Redo
Buffer Pool
Snapshot of T4
T2
T4
T5
T1
T3
T3T2T1 T4 T5
T3T2T1 T4
T1
T4Buffer Pool
Data
Redo Log
Snapshot Version
T4
Unflushable
T5
T3T2T1 T4
Flushable
T4T3T2Primary
Snapshot Version
T4
LSN of the latest
applied redolog
RO Node
RO Node

15. Single Master
Single Endpoint Transparent Failover
Attacks Protection Causal Consist Read
Proxy Cluster
Master Replica Replica
Shared Storage
Application
Replica
Read/Write Split
High Availability
Load Balance
Security

16. Read and Write Separation - Session Consistent
Problem Can’t read latest data Solved!
connection.query
{
UPDATE user SET name=‘Jimmy’ WHERE id=1;
COMMIT;
SELECT name FROM user WHERE id=1; // name is Jimmy
}
SELECT can always get the latest data
POLARDB
Cluster
LSN 30 LSN 35
1. UPDATE
2. SELECT
Log Serial Number
LSN 35
1. UPDATE 3. SELECT Require LSN>=35)
2. Return LSN=35
M R1 R2
Application
Smart
Proxy
Read & Write
Separation
Load Balance
Module

17. Multi-master
Storage
Monitoring
Segment Servers
DB Servers DB ServersDB Servers
AZ1 AZ2 AZ3
segment 1
Segment Servers Segment Servers
CS1 CS2 CS3 CS4 CS5 CS6
Database Cluster
(Compute)
Storage
Service
3-AZ Persistent Core
(append-only)
TCPTCP
Storage Cluster
Page read
RDMA RDMA RDMA
redo
segment 1 segment 1

18. Thank You