6 juin 2015 Samedi SQL à Québec Session 3 - Data (SQL Azure, Table et Blob Storage) (Eric Moreau) SQL Azure est une base de données relationnelle en tant que service, Azure Storage permet de stocker et d'extraire de gros volumes de données non structurées (par exemple, des documents et fichiers multimédias) avec les objets blob Azure ; de données NoSql structurées avec les tables Azure ; de messages fiables avec les files d'attente Azure.

1. Azure Data Overview
Eric Moreau, MVP
www.emoreau.com – eric@moer.ca
Microsoft Azure

3. SQL
Database
SQL Server
VM
DocumentDB
Search HDInsight
Additional
databases
Agenda – Part 1

4. Agenda – Part 2
Blobs Files
Queues Tables

5. Agenda – Part 3 – Announced in May 2015
Data Lake
Azure SQL
Data
Warehouse

9. SQL
Database
SQL Server
VM
DocumentDB
Search HDInsight
Additional
databases
Microsoft Azure Data Platform

10. Run SQL on VM
Run any SQL product on cloud VM
Support for SQL Server, Oracle, MySql
Ready to go VM images available in Gallery
Persistent storage using attached disk in blob
storage (Azure Files)
Has all the benefits and powers of VMs combined
with the full features of a SQL Engine

11. SQL
Database
SQL Server
VM
DocumentDB
Search HDInsight
Additional
databases
Microsoft Azure Data Platform

12. A Server is not a machine
TDS = Tabular Data Stream

13. SQL Server database technology “as a Service”
Fully Managed (PaaS)
Enterprise-ready with automatic support for HA, DR,
Backups, replication and more
SQL Database – The Basics

14. Scale out with ElasticScale
Built-in regional database replicas for additional
protection
Uptime SLA of 99.99% (4.38 minutes / month)
SQL Database – The Basics

15. ElasticScale

16. • New portal
• Some features in preview
• Full Text Search
• Column Store Index
• Users in contained database (+row-level security)
• Parallel queries (Premium only)
• CLR Integration
• T-SQL window functions (over clause)
• Change tracking
SQL Database – v12

17. Service
Tier
Performance
Level
Common App
Pattern
Performance Business Continuity
Max DB
Size
Transaction Perf.
Objective
DTU PITR DR / GEO-Rep
Basic Basic Small DB, SQL opp 2 GB Reliability / Hr. 5 7 Days DB Copy +
Manual Export
Standard S0
S1
S2
Wrkgp/cloud app,
multiple concurrent
operations
250 GB Reliability / Min. 10
20
50
14 Days DB Copy +
Manual Export
Premium P1
P2
P3
Mission Critical, High
volume, Many
concurrent Users
500 GB Reliability / sec. 100
200
800
35 Days Active Geo-
replication
Selecting the right SQL Database edition
This information is subject to change over time.

18. Blended Measure
Relative

19. SQL Database SQL IaaS
Why (at least) two offerings of SQL in Azure?

20. SQL Database (PaaS) SQL in a VM (IaaS)

21. SQL
Database
SQL Server
VM
DocumentDB
Search HDInsight
Additional
databases
Microsoft Azure Data Platform

22. Fully managed, scalable JSON
document database service

23. Fully Managed NoSQL Document-based Database Service
Automatic and consistent indexing of all properties
Tunable High Performance
Backed by write-optimized SSD
Easy to scale
Native Support for JSON and JavaScript
Designed cloud first

24. A document store
Collections
Request
Document 1
{
"name": "John",
"country": "Canada",
"age": 43,
"lastUse": "March 4, 2014"
}
{
"name": "Eva",
"country": "Germany",
"age": 25
}
Document 2
{
"name": "Lou",
"country": "Australia",
"age": 51,
"firstUse": "May 8, 2013"
}
Document 3
{
"docCount": 3,
"last": "May 1, 2014"
}
Document 4
{…}
DocumentDB
Application

25. DocumentDB Overview
A NoSQL document database-as-a-service, fully managed by Microsoft Azure.
For cloud-designed apps when query over schema-free data; reliable and predictable performance; and rapid
development are key. First of its kind database service to offer native support for JavaScript, SQL query and
transactions over schema-free JSON documents.
Perfect for cloud architects and developers who need an enterprise-ready NoSQL document database.
Release Timing
Public Preview: August 21, 2014
GA: H2 FY15

26. Proven @ Scale
Data store powering production Microsoft services.
 Scale tested with databases over 10TBs of JSON document
data from Office OneNote API service
 Rapidly evolving data models and query patterns based on
3rd party applications
 Serving over 1 million active users
 Operating consistently with 99.95% availability
First Party Use Cases
 Store, roam and query user generated data across
applications
 Extensible data models for 3rd party integration
 Social activity and awareness feeds with views and filtering

27. Programmability
Broad language and platform support
 REST/HTTP APIs
 C#, Node.js, JavaScript and Python SDKs
for Public Preview
 SDKs open sourced through GitHub
C# Client SDK
 Supports gateway and direct connectivity
 Async APIs for all operations
 HTTP and TCP transports available
 POCOs, inherited document types and dynamics
 LINQ provider for query execution
client = new DocumentClient(new Uri(endpointUrl), authKey);
Database database = await client.CreateDatabaseAsync(new
Database{Id = “myNewDatabase”});
DocumentCollection collection = await
client.CreateDocumentCollectionAsync(database.SelfLink, new
DocumentCollection{Id = “Pages"});
Page page = new Page
{
Id = "Sample",
Title = "About Paris",
Language = new Language { Name = "English" },
Author = new Author { Name = "Don” },
Content = "Don's document in DocDB is a valid JSON document",
PageViews = 10000,
Topics = new Topic[] {
new Topic { Title = "History" },
new Topic { Title = "Places to see" }}
};
ResourceResponse<Document> resp = await
client.CreateDocumentAsync(collection.DocumentsLink, page);

28. SQL
Database
SQL Server
VM
DocumentDB
Search HDInsight
Additional
databases
Microsoft Azure Data Platform

29. Data size
Access
Updates
Structure
Integrity
Scaling
Relational DB vs. Hadoop

30. SQL
Database
SQL on IaaS DocumentDB
Search HDInsight
MySQL,
Oracle
Microsoft Azure Data Platform

31. Additional Database options in Azure
Oracle VMs are supported in Azure.
MySQL is offered from the partner ClearDB.

32. Agenda – Part 2 - Storage
Blobs Files
Queues Tables

33. Microsoft Azure
Storage Blob

34. Two Types of Blobs Under the Hood

35. Two Types of Blobs Under the Hood
Block Blob:

36. Two Types of Blobs Under the Hood
Page Blob:

37. Blob Storage Concepts

38. Microsoft Azure
Storage Files

39. Azure Files – Usage
Share data across VMs and applications
Share settings throughout services
Dev/Test/Debug

40. Azure Files
Shared Network File Storage for Azure
Availability, durability, scalability are managed automatically
Supports two interfaces: SMB and REST

41. “I wish I could go to storage and provision a cloud drive, giving it a
namespace, and that drive would then be UNC-addressable by the
OSes.”
“I need two VM's running with a shared drive. One will write to the
drive, the other will read [it].”
“Hi, I have two VM's in Microsoft Azure. All I want to do is set up a file
share between them. Is this possible?”
“Is it possible to share a secondary disk between different VM
instances?”
Azure Files – Customer Quotes

42. Microsoft Azure
Storage Queue

43. Producers Consumers
Queue C1
C2
P1
P2
4 3 12
Queue-based Load Levelling Pattern

44. Queue length reflects how well the backend processing
nodes are doing.
Decouples the application.
Flexibility of efficient resource usage within an
application.
Absorb traffic bursts and reduce the impact of individual
component failures.
Why use a Queue?

45. Messages are ordered but not guaranteed FIFO.
Message will be processed at least once.
Message may be processed more than once.
.DequeueCount increases every time.
-> Processing must be idempotent.
Queue Considerations

46. Messages are stored up to 7 days
Event buses are built on top of queues
Queue Considerations

47. Microsoft Azure
Storage Table

48. Table Storage Details

49. Table Storage Concepts

50. No Fixed Schema

51. Querying

52. Table Storage Details

53. Table Storage Details – Entity Properties

54. A key/value store
Tables
Partition
B
Partition
A
Property
Entity
2B
Property Type
Data
Property Name
String intString Date
Name LastUseCountry Age
String intString
Name Country Age
String intString Date
Name FirstUseCountry Age
Row key
1
2
1
2Partition key
A
A
B
B
int Date
LastCount
2B
Azure Storage Tables
Application

55. To recap Azure Storage
Tables
Scalable service for non-relational data
Blob Storage
Storage of files, VHDs, mp4s, pngs, etc…
Files
SMB File Sharing
Queues
Low latency message processing

56. Azure Data Lake

57. Azure Data Lake

58. Azure Data Lake

59. Microsoft Azure
SQL Data
Warehouse

60. Azure SQL Data Warehouse

61. Azure SQL Data Warehouse

62. SQL Database
(PaaS)
SQL Server
VM (IaaS)
DocumentDB
Search HDInsight
MySQL,
Oracle
Microsoft Azure Data Platform - Recap

63. Relational
database in the
cloud (PaaS)
SQL Server
VM (IaaS)
DocumentDB
Search HDInsight
MySQL,
Oracle
Microsoft Azure Data Platform - Recap

64. Relational
database in the
cloud (PaaS)
Full relational
database in the
cloud (IaaS)
DocumentDB
Search HDInsight
MySQL,
Oracle
Microsoft Azure Data Platform - Recap

65. Relational
database in the
cloud (PaaS)
Full relational
database in the
cloud (IaaS)
NoSQL – Json
documents
Search HDInsight
MySQL,
Oracle
Microsoft Azure Data Platform - Recap

66. Relational
database in the
cloud (PaaS)
Full relational
database in the
cloud (IaaS)
NoSQL – Json
documents
Search Big Data
MySQL,
Oracle
Microsoft Azure Data Platform - Recap

67. Relational
database in the
cloud (PaaS)
Full relational
database in the
cloud (IaaS)
NoSQL – Json
documents
Search Big Data
Any other
database in the
cloud
(IaaS/PaaS)
Microsoft Azure Data Platform - Recap

68. Blobs Files
Queues Tables
Microsoft Azure Data Platform - Recap

69. Huge storage
Block = Streaming
Page = Random access
Files
Queues Tables
Microsoft Azure Data Platform - Recap

70. Huge storage
Block = Streaming
Page = Random access
Shared access
between
services
Queues Tables
Microsoft Azure Data Platform - Recap

71. Huge storage
Block = Streaming
Page = Random access
Shared access
between
services
Asynchronous
processes Tables
Microsoft Azure Data Platform - Recap

72. Huge storage
Block = Streaming
Page = Random access
Shared access
between
services
Asynchronous
processes
NoSQL –
Key/Value pair
Microsoft Azure Data Platform - Recap

73. Data Lake
Azure SQL
Data
Warehouse
Microsoft Azure Data Platform - Recap

74. fully featured RDBMS
transactional processing
rich query
managed as a service
elastic scale
internet accessible http/rest
schema-free data model
arbitrary data formats
Microsoft Azure Data Platform - Recap

75. Azure Data Overview
Eric Moreau, MVP
www.emoreau.com – eric@moer.ca
Microsoft Azure