NoSQL databases are everywhere and they're here to stay. For a successful scalable web application, it is vital to understand the capabilities of NoSQL databases. Relational databases are not dead, either, and they will not be. Understanding the abilities of each will help you to make better architectural decisions. Hybrid database solutions could use the best of the two worlds. In this talk, we will delve into the important decision making factors for each. As an example, we will study an online game at Ubisoft Montreal that switched from a SQL database to a polyglot persistence with no data loss and no downtime.

1. Polyglot
Databases
php[tek] 2015

2. @majidfn
Majid Fatemian
Ubisoft Montreal

3. Relational

4. DATA

5. BIG
DATA

6. NoSQL

7. 1 2 3
Non-Relational Cluster Schemaless

8. 1. Document
2. Key-Value
3. Graph
4. Column
MongoDB, CouchBase
Memcache, Redis
Neo4J, OrientDB
Cassandra, HBase

9. A
C P
Availability
Consistency Partition Tolerance

10. Consistency vailability
A
artition
PConsistency
vailability
A artition
P
MongoDB
MySQL
Postgres
CouchBase
Cassandra
Riak

11. Fitness Evolved

12. Fitness Evolved

13. Fitness Evolved

14. Fitness Evolved

15. 3M 15K 160K
Players CCU Daily Ses.
STATS

16. 2011
2012
2013
2014

17. 2011
2012
2013
201

19. Application

20. Normalized
ACID
Transaction{Consistency
Relational

21. Replication
Table Partitioning
Availability
{Relational

22. OPs / DBAs
Tools
Devs{DevOps
Relational

23. 2011
2012
2013
201

24. Aggregated
…
Detailed
Profile
Community
History
Leaderboards

25. Single Point Failure Availability
} Relational

26. Scalability
Relational

27. Schema
Relational
{
player_id: 1234,
* calories: 40,
* duration: 75,
* activity: jumping_ropes,
game_id: 4001,
platform: WiiU,
difficulty_level: medium,
repetition: 178,
score: 450,
endurance:3
}
{
player_id: 1234,
calories: 75,
duration: 120,
activity: boxing
}

28. Player Game Cals Duration Activity Difficulty Score Reps.
1000 1 100 80 Box - - -
2000 2 90 100 Jump H 4.5 180
1000 2 80 120 Pushup M 4 130
2000 1 120 110 Situps - - -

29. Player Game Cals Duration Activity Difficulty Score Reps.
1000 1 100 80 Box - - -
2000 2 90 100 Jump H 4.5 180
1000 2 80 120 Pushup M 4 130
2000 1 120 110 Situps - - -

30. Distribution
Relational
Data C
Data B
Data A
Data Access Layer

31. ACID
Transaction
Partitioning
Replication
Tools
OPs
Data Volume
Fixed Schema
Scale Up
Manual Sharding
+ -

32. NoSQL
...

33. mongoDB
1 2 3
OPs DEVs Community

34. Schemaless
Non-Relational
player_id: 1234,
* calories: 40,
* duration: 75,
* activity: jumps,
game_id: 4001,
platform: WiiU,
difficulty_level: 2,
repetition: 178,
score: 450,
endurance:3
player_id: 1234,
calories: 75,
duration: 120,
activity: boxing

35. Scalability
Non-Relational

36. Scalability
Non-Relational
+

37. Sharding
Non-Relational

38. Sharding
Non-Relational
Data A
A
Data B
B Data CC

39. •DB Managed
•Balanced Load
Sharding
Non-Relational
•Sharding key
•Bouncing
•Decreased resilience

40. Map/Reduce
Non-Relational
R1 R2 R3

41. Map/Reduce
Non-Relational
QueryQueryQuery
R1 R2 R3

42. Map/Reduce
Non-Relational
QueryQuery Query
R1 R2 R3
Map

43. Map/Reduce
Non-Relational
QueryQuery Query
R1 R2 R3
Result
ReduceMap

44. Denormalized
Non-Relationalread /
}write
Disk Space

45. Query
Non-RelationalData Model
Aggregation
}

46. Eventual-
Consistency
Non-Relational
X1
X1 X1

47. Eventual-
Consistency
Non-Relational
X2
W
X1 X1
X2

48. Eventual-
Consistency
Non-Relational
X2
W
X1 X1
R
X1
X2

49. Eventual-
Consistency
Non-Relational
X2
W
X2
X2
R
X2
X2

50. Locking
Non-RelationalDB Level (2.2)
}Document Level (3.0)

51. Backup
Non-Relational
Point-in-time

52. Denormalized data
Disk Space
Expertise
Complex Querying
Eventual Consistency
Resource Usage
DB-Level Locking
-
Schema-Less
Aggregated Data
Large Scale Data
Sharding
Map/Reduce
Memory Storage
Journaling
+

53. Random Profile
Search
(1M)
Full History
Retrieval
(6M)
Insert
(6M)
Aggregation
Map/Reduce
(6M)
0
0.003
0.005
0.008
0.01
MySQL
MongoDB

54. Persistence
Polyglot

55. Profile (Total Sum)
Leaderboards
…
Activity Details
Activity Aggregations

56. Master
Slave
Config Server
Shard 1
Replicaset
Shard 2
Replicaset

57. Data Access Layer

58. The
Challenge

59. Inconsistency1

60. Data Access Layer
Commit
2-Phase

61. Data Access Layer
W
Pending
Commit
2-Phase

62. Data Access Layer
Pending
W
Commit
2-Phase

63. Data Access Layer
Done
Commit
2-Phase

64. Consistency
vs.
Latency
2

65. Operations3

66. Availability4

67. Data Access Layer

68. Data Access Layer

69. Data Access Layer

70. Data Integrity5

71. Unit
{TEST Integration
Functional

72. DATA
Migration

74. Profile
Activity }3rd Party
{
profile:
{
GUID:…,
platform:
{
name:’xbox’,
xuid:…
}
},
activity:…,
duration:…,
datetime:…
}

75. 1 One-Shot Migration
2 Profile Versioning
4
hrs
!!
Live
!!

76. +1

77. R
W
V
+1
1

78. R
W
V
+1
2

79. Final
Results

80. 2011 2012 2013 2014

81. 9x response time

82. 3 years uptime

83. > Go Hybrid, wisely!

84. > Go Hybrid, wisely!
Denormalized data
Eventual Consistency
Evolving
Schema-Less
Large Scale Data
Sharding
Map/Reduce
ACID
Transaction
Replication
Tools, OPs
Data Volume
Fixed Schema
Scaling

85. Thank
YOU!
@majidfn
http://joind.in/13728