Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Azure Storage Performance

1 500 vues

Publié le

source code for demonstration
https://github.com/TeamSpark-Learning/Storage-Node-01

Publié dans : Logiciels
  • Soyez le premier à commenter

Azure Storage Performance

  1. 1. Azure Storage Anton Boyko Microsoft Azure MVP boyko.ant@live.com
  2. 2. Agenda Azure Storage • Overview • Blob Storage • CDN • Drive Storage • Queue Storage • Table Storage • File Storage Azure Storage Performance • Blob Storage • Drive Storage • Table Storage
  3. 3. OVERVIEW
  4. 4. Storage in the cloud • Exposed via RESTful web services • Available globally
  5. 5. Storage SDK in many languages C#/.NET Python Ruby Perl Node.JS Java PHP Erlang Common LISP Objective- C
  6. 6. Storage security • HTTP or HTTPS • Two independent 512bit symmetric master keys • Shared access signatures for more granular access
  7. 7. Storage artifacts Blobs Drives Queues Tables Files
  8. 8. BLOBS
  9. 9. Blob storage concept Block/Page Blob Container Account contoso image pic01.jpg block01 block02 pic02.jpg video mov01.avi page01 page02 https://contoso.blob.core.windows.net/image/pic01.jpg
  10. 10. Blobs main operations • Get • Put • Delete • Copy
  11. 11. Quiz time Question Why do I need a “copy” command if it will do pretty much the same as “get” and “put” do together? Answer
  12. 12. Quiz time Question Why do I need a “copy” command if it will do pretty much the same as “get” and “put” do together? Answer “Copy” command is server- side copy. I do not need to download and upload my blob. Also it’s possible to use different storage accounts for source and destination (even in different locations).
  13. 13. Blobs main operations • Get • Put • Delete • Copy • Snapshot • Lease
  14. 14. Blob types Streaming workloads Block Random access workloads Page
  15. 15. Block blobs Benefit Efficient continuation and retry Parallel and out of order upload of blocks
  16. 16. Page blobs 0 10 GB 512 1024 1536 2048 2560
  17. 17. Quiz time Question How can we increase upload speed for page blobs? What is the preferred way to upload 1TB VHD to Azure? Answer
  18. 18. Quiz time Question How can we increase upload speed for page blobs? What is the preferred way to upload 1TB VHD to Azure? Answer Upload in many threads. Upload only non-zero pages.
  19. 19. CONTENT DELIVERY NETWORK
  20. 20. Content Delivery Network • High-bandwidth global content delivery • Many origins – Web App – Cloud Service – Storage Account
  21. 21. Quiz time Question Can I use CDN for my VM’s in Azure? Answer
  22. 22. Quiz time Question Can I use CDN for my VM’s in Azure? Answer • Yes. All VMs sit behind Cloud Services.
  23. 23. Content Delivery Network • High-bandwidth global content delivery • Many origins – Web App – Cloud Service – Storage Account • New origin – Custom URL
  24. 24. http://sally.blob.core.windows.net/images/pic1.jpg http://sally.blob.core.windows.net/  http://guid01.vo.msecnd.net/ Edge Location Edge Location Edge Location
  25. 25. Content Delivery Network
  26. 26. DRIVES
  27. 27. Azure Drives • Durable NTFS volume for Azure Instances – Use existing NTFS APIs to access a network-attached durable drive – Use System.IO from .NET • Benefits – Move existing apps that are using NTFS more easily to the cloud – Durability and survival of data on instance recycle – Drives can be up to 1TB • Azure Drive is NTFS VHD Page Blob – Mounts Page Blob over the network as NTFS drive – Local cache on instance for read operations – All flushed and unbuffered writes to drive are made durable to the Page Blob
  28. 28. TABLES
  29. 29. Table storage concept Entity Table Account contoso users FName=Bob LName=Smith FName=Inga Phone=123 groups Name=Admin
  30. 30. Table details Table • Create • Query • Delete Entity • Insert • Update – Merge – Replace • Upsert • Query • Delete
  31. 31. Entity properties • Entity can have up to 255 properties – Up to 1MB per entity • Mandatory Properties for every entity – PartitionKey & RowKey (only indexed properties) • Uniquely identifies an entity • Defines the sort order – Timestamp • Optimistic Concurrency • Exposed as an HTTP Etag • No fixed schema for other properties – Each property is stored as a <name, typed value> pair – No schema stored for a table – Properties can be the standard .NET types String, binary, bool, DateTime, GUID, int and double
  32. 32. Purpose of Partition Key • Entity Locality – Entities in the same partition will be stored together – Efficient querying and cache locality – Endeavour to include partition key in all queries • Entity Group Transactions – Atomic multiple Insert/Update/Delete in the same partition in a single transaction • Table Scalability – Target throughput – 2 000 tps/partition, 20 000 tps/account – Azure monitors usage patterns of partitions – Automatically load balance partitions – Each partition can be served by a different storage node – Scale to meet traffic needs of your table
  33. 33. Poker scoring table By players • PK = PlayerID • RK = GameID • Score = 42 By games • PK = GameID • RK = PlayerID • Score = 42 there is no silver bullet
  34. 34. Blog tables Posts • PK = Date • RK = Time • Authors • Tags • Text PostsByAuthors • PK = UserID • RK = Post PK + RK PostsByTags • PK = Tag • RK = Post PK + RK
  35. 35. IT’S SHOW TIME
  36. 36. DRIVES
  37. 37. Single drive
  38. 38. 4 * 1/4 drives
  39. 39. BLOBS
  40. 40. Upload single-thread Mbytes uploaded: 142.49238300323486 total time in ms: 7377 avg ms per Mbyte: 51.771188357714024 Mbytes uploaded: 1424.9273290634155 total time in ms: 70467 avg ms per Mbyte: 49.45304828023543
  41. 41. Upload multi-thread total files uploaded: 365 total Mbytes uploaded: 142.4734115600586 total time in ms: 4576 avg ms per Mbyte: 32.11827350727137 total files uploaded: 365 total Mbytes uploaded: 1424.9266424179077 total time in ms: 36323 avg ms per Mbyte: 25.491136819762723
  42. 42. Copy blob Mbytes uploaded: 127000 total time in ms: 177 avg ms per Mbyte: 0.0013937007874015748 Mbytes uploaded: 127000 total time in ms: 1836 avg ms per Mbyte: 0.014456692913385827
  43. 43. TABLES
  44. 44. Insert single entity records inserted: 500 total time in ms: 2920 avg ms per record: 5.84
  45. 45. Insert batch of 50 records inserted: 500 total time in ms: 470 avg ms per record: 0.94
  46. 46. Insert batch of 100 (9100 times) records inserted: 910000 total time in ms: 1072417 (~18 min) avg ms per record: 1.1784802197802198
  47. 47. Select all single-thread records selected: 1500 queries count: 2 total time in ms: 274 avg ms per record: 0.18266666666666667
  48. 48. Select single partition records selected: 5000 queries count: 5 total time in ms: 815 avg ms per record: 0.163
  49. 49. Select all single-thread records selected: 1142100 queries count: 1143 total time in ms: 335824 (~6 min) avg ms per record: 0.29404080203134575
  50. 50. Select all multi-thread records selected: 1142100 queries count partition: 306 queries count data: 1259 total time in ms: 48134 (~48 sec) avg ms per record: 0.04214517117590403 total http queries: +37% total time in ms: -76% avg ms per record: -76%
  51. 51. Select by custom property records selected: 284927 queries count: 285 total time in ms: 74065 avg ms per record: 0.2599437750722115
  52. 52. Questions Anton Boyko Microsoft Azure MVP boyko.ant@live.com facebook.com/boyko.ant youtube.com/user/boykoant Google: boykoant

×