Managing terabytes on aws
•
2 j'aime•994 vues
JFrog is handling huge amount of binaries files for all our customers. Since each customer has its own space and domain, using a global Object Store can be tricky. Also our application cannot work with an “eventually consistent” storage, and cannot deliver customer requirements with current S3 performance. Learn in this session how we managed fast upload, critical replication and backups, and global download availability of the terabytes of JFrog customer binaries files.
Recommandé
Contenu connexe
Similaire à Managing terabytes on aws
Plus de Baruch Sadogursky
Plus de Baruch Sadogursky (20)
Dernier
Dernier (20)
Managing terabytes on aws
- 13. Request Storage File Listing Checksum Storage
- 14. Request Storage File Listing Checksum Storage Metadata requests Cache check, FS request DB request
- 15. Request Storage File Listing Checksum Storage Metadata requests Cache check, FS request DB request HEAD requests Cache check, FS request DB request
- 16. Request Storage File Listing Checksum Storage Metadata requests Cache check, FS request DB request HEAD requests Cache check, FS request DB request GET request Cache check, FS request DB + FS request
- 17. Request Storage File Listing Checksum Storage Metadata requests Cache check, FS request DB request HEAD requests Cache check, FS request DB request GET request Cache check, FS request DB + FS request PUT request FS request DB + FS request
- 38. EBS + LVM = WIN!
- 40. /data 250GB 250GB 250GB 250GB
- 41. /data 250GB 250GB 250GB 250GB 250GB 250GB
- 51. Backup files
- 53. Backup files Backup database Keep them in sync!
- 54. Backup files Backup database Keep them in sync!
- 56. Backup only deltas Manage threads to ec2 efficiently
- 57. Backup only deltas Manage threads to ec2 efficiently Keep the data isolated
- 58. Backup only deltas Manage threads to ec2 efficiently Keep the data isolated
- 74. Create empty Artifactory instance
- 75. Create empty Artifactory instance Populate from back storage on demand
- 76. Create empty Artifactory instance Populate from back storage on demand Drink Cosmopolitan
- 77. Conclusions time 1. Be lazy.
- 79. Conclusions time 1. Be lazy. 2. Be smart.
- 80. Conclusions time 1. Be lazy. 2. Be smart. 3. Be agile.