Norway VMUG Tour - The Architecture Behind Policy-Driven Data Protection - A Rubrik Deep Dive

1. The Architecture Behind
Policy-Driven Data Protection
A Rubrik Deep Dive
Norwegian
Tour

2. Chris Wahl
Writer
Co-Host
Instructor
Chief Technologist
Author
Microsoft
VMware
@ WahlNetwork.com
@ DatanautsPodcast.com
@ Pluralsight.com
@ Rubrik.com
Networking for VMware Admins
MVP (PowerShell)
VCDX #104
(Data Center & Network Virtualization)
@ChrisWahl | chris.wahl@rubrik.com

3. Today’s Topics
•An Introduction to Rubrik
–Cloud Data Management
–Understanding the Distributed Architecture
–Snapshot Chains & Immutability
•The “vSphere DRS” for Data Protection
–Conceptual Design Walkthrough
–SLA Domains for Policy Driven Protection
–A Technical Look at Cerebro

4. An Introduction to
Rubrik’s Distributed
Architecture

5. Stop Being a Mine Sweeper – 2014 Presentation
https://www.slideshare.net/ChristopherWahl

6. Data Management: 1990s to Present
1990s – Present
Backup &
Replication
Software
Backup Storage
Backup
Software
Backup
Servers
Backup
Proxies
Replication Catalog
Database
Tape Off-site ArchiveBackup Storage
a
Dedupe
Metadata
2000s – Present
Data Management: 2000s to Present

7. 7

8. “A lot of stuff we do today is
because of legacy”
Per Buer, CEO at IncludeOS

9. 9

10. Meet Rubrik Cloud Data Management
Backup
Software
Backup
Servers
Backup
Proxies
Replication Catalog
Database
Tape Off-site ArchiveBackup Storage
a
Dedupe
Metadata
Private Public
Software fabric for orchestrating apps and data across clouds. No forklift upgrades.
EDGE

11. How It Works
Quick Start: Rack and go. Auto-discovery.
Automate: Intelligent SLA policy engine for
effortless management.
Rapid Ingest: Flash-optimized, parallel ingest
accelerates snapshots and eliminates stun.
Content-aware dedupe. One global namespace.
Instant Recovery: Live Mount VMs & SQL.
Instant search and file restore.
Secure: End-to-end and Data at Rest encryption.
Immutability to fight Ransomware.
Cloud: “CloudOut” instantly accessible with
global search. Launch apps with “CloudOn” for
DR or test/dev. Run apps in cloud.
Primary Environment
SLA Policy Engine
Log Management
Private Public
NAS
AHV Hyper-V
VMware VMwareVMware VMwareVMware VMware

12. Distributed File System
……
Atlas

13. Distributed Architecture
Atlas
Atlas
Atlas Atlas
Application
Metadata
(Callisto)

14. Scalable Architecture
14
Atlas
Atlas
Atlas Atlas
Application
/sd
Atlas
Atlas
...
...
Atlas
Atlas
Atlas
Atlas
Metadata

15. Self Healing
15
/sd/foo/bar

16. Snapshot Chains
16
Full
Time
t0
Incrementals
t1 t3t2 t4
materialize

17. Snapshot Chains (Redirect-on-Write)
17
log file ...
immutable

18. status : QUEUED
startTime : 2017-10-13T19:12:30Z (12:12:30 PST)
id : MOUNT_SNAPSHOT_811272f6-712d-4ccc-8695-132297c97fa7
http://wahlnetwork.com/2017/10/16/building-live-mount-rubrik/

20. Meet Cerebro:
The Brains Behind
Rubrik

21. Top Architects
Transform Complexity Into
Simplicity

23. Conceptual Design
Consistent
data
Disaster!
Systems
recovered
Services
resume
RPO RTO WRT
MTD

24. Business Language
•Service Level Agreement
–An understanding between two parties for data protection
•A layer of abstraction that declares intent
–Inform the fabric what you want
–Let the fabric determine how to implement your end-state
•Should be declarative in nature
–Describe an end state and allow a machine learning (ML) algo to execute

25. Policy{SLA

26. Recovery Point Objective (RPO)
Availability Duration (Retention)
When to Archive (RTO)
Replication Schedule (DR)
{SLA

27. There’s no need
for you to build
backup jobs

30. Apply policy to
VMs, folders,
clusters, data
centers, vCenter,
DBs, servers,
filesets, NAS …

31. Inheritance
VM VM VMVM
Test
Silver Policy Assignment
VM VM VMVM
Production
Gold Policy Assignment
VM
VM VMVM VM

32. VM VM VM
Test
Silver Policy Assignment
VM VM VMVM
Production
Gold Policy Assignment
VM
VM VMVM VM
Direct Assignment
Silver Policy Assignment
VM

33. vSphere Categories & Tags

34. vSphere Categories & Tags

35. The Choice is Yours
World Class UI Experience Full Featured RESTful API
• Entirely driven by HTML5
• Simple to learn
• Consumer grade polish
• Great for ad-hoc activities run by
administrators or end users
• If you really liked Java, I’m sorry
• Entirely driven by the Open API
Spec 2.0 (fka Swagger)
• Simple to learn
• Gracefully integrate with your
existing toolset
• Tested and validated by the core
engineering team – we use it, too!
• No feature parity between the UI
and API
35

36. Rubrik is a Programmatic Software Fabric
delete
post
get
put
Your App
Automation
Create, select, execute. Repeat.
Extensible
APIs designed to be resilient to
change.
API-first Architecture
Rubrik consumes the same APIs.
patch

37. Distributed Task
Scheduler

38. Under the Hood
“The Interface”
“The Logic”
“The Core”
Distributed Task Framework
Callisto
Distributed Metadata Store
Cluster Management
Global Search
Cerebro
Data Management
Crystal
UI / API
Infinity
Ecosystem
Integration
Thor
Cloud Connect
Atlas
Cloud-Scale File System
NFS

39. Distributed Task Scheduler
•Master-less architecture
•Distributed task queue
•No single point of failure
•Leverages metadata service
•Parallel processing of smaller tasks
•Nodes independently decide which
tasks to run
•Auto load distribution
Reports
Full Backup
Incremental
Archive
Replicate
Expire
Distributed
Task Scheduler
Index
Mount
VM

40. It’s like vSphere DRS
for Data Protection

41. How It Works
•Mutual exclusion: No two threads will concurrently perform actions for the
same task instance at the same time.
•Progress: All task instances in active states with be acted upon promptly
once the system is in a stable state.
•Terminal permanence: No side effect of an active state of a task instance
can ever be observed after that instance moves to a terminal state.

42. Summary

43. Backup
Replication
Archival
Analytics
Cloud-Native
Applications & Data
Run Rubrik Anywhere – Data Center to Cloud
On-Prem
Applications & Data
Azure InstanceEC2 Instance

44. Rubrik Manages All Your Data – Anywhere
NAS
Production Rubrik Cloud Data Management Platform
Backup &
Recovery
Disaster
Recovery
Archival &
Compliance
Search
Copy Data
Management
Analytics
Automated,
secured data
management
anywhere.
EDGE

45. Rapid Technology Innovation + Market
Recognition
v1.0
Hybrid Cloud
Backup & Recovery
v2.0
Offsite/Cloud
Replication/DR
v2.2
Security & Scale
v2.1
Auto-Protect
v2.3
Object Store
Support Expansion
v3.0
Physical & Edge
v3.2
Cloud Cluster
v3.1
Rich Visual Reporting
(Envision)
v4.0
All Workloads,
Anywhere
Magic Quadrant for
Data Center Backup
and Recovery
Solutions
2015 2016 2017

46. Thank You!
@ChrisWahl

47. Vester - Bonus Content!
•Are you a VMware administrator?
•Can you answer these questions, with certainty, without looking?
–Do all of your hosts have the same NTP settings?
–How aggressive is your cluster’s DRS migration policy?
–Would any of your VMs fail a vMotion because an ISO file is currently mounted?
–What’s the oldest VM snapshot you have hanging around?
•Wouldn’t you like to know?
vSphere Configuration Management - https://github.com/WahlNetwork/Vester

48. YouTube - Watch Session #VMTN6639U