SlideShare une entreprise Scribd logo
1  sur  20
Amazon Web Services

     Jared Rosoff
      @forjared
Agenda
• Single node
  – Instance sizing
  – Storage configuration
  – OS configuration
• Replica Set
  – Availability Zones
  – Disaster Recovery
  – Multi Data-Center
• Sharded Cluster
INSTANCES, STORAGE AND SINGLE
NODE CONFIGURATION
Instance Sizing
              32-bit = Don’t Use

                       Small

                  Arbiter / Config

                        Big

              32-bit = Don’t Use


                       Huge
OS
• Amazon OS now an option
• Turn off atime
• Raise file descriptor limits
    cat >> /etc/security/limits.conf << EOF
    * hard nofile 65536
    * soft nofile 65536
    EOF
• DO NOT use large VM pages
• Use ext4, xfs
• Use RAID
    – RAID10 on MongoD
    – RAID1 on ConfigbDB

• Warning! Known problems with Ubuntu 10.04 &
  EBS
    –   https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853
    –   https://bugzilla.kernel.org/show_bug.cgi?id=16991
A MongoDB Data Node
 This is your typical database node. Used for actual data
 storage. We’ll use this icon to represent this type of node
                   throughout this prezo.


                                 •   64-bit instance
mongod
                                 •   More RAM == better
                                 •   Run EXT4 or XFS file system
                                 •   Turn off atime & diratime
              EBS – RAID10
                                 •   EBS volumes in RAID10
A MongoDB Config Server
      Used to store configuration data about sharded
deployments. Deployed as a set of 3 servers. Relatively low
             load and storage requirements.


                               •   64-bit instance
                                    – Micro is fine
mongoc
                               •   EBS volumes in RAID1

              EBS – RAID1
A MongoDB Arbiter
      Used to store configuration data about sharded
deployments. Deployed as a set of 3 servers. Relatively low
             load and storage requirements.


                               •   Micro is fine
mongod --
                               •   No storage requirements
 arbiter                       •   Must be separate node from rest
                                   of replica set
REPLICA SET DESIGNS
Single Region Replica Set
                                      Region

  Availability Zone A          Availability Zone B          Availability Zone C

mongod                       mongod                       mongod


              EBS – RAID10                 EBS – RAID10                 EBS – RAID10
Disaster Recovery Site
                        Region 1                                       Region 2

  Availability Zone A              Availability Zone B          Availability Zone A

mongod                         mongod                         mongod

                                                              Hidden
              EBS – RAID10                     EBS – RAID10                  EBS – RAID10
Multi Data Center
       Region 1                       Region 2                    Region 3

  Availability Zone A           Availability Zone A          Availability Zone A

mongod                       mongod                        mongod

                             Priority 1
              EBS – RAID10                  EBS – RAID10                 EBS – RAID10
SHARDED CLUSTERS IN EC2
Single Region Sharded Cluster
                                      Region

  Availability Zone A          Availability Zone B          Availability Zone C

mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10


mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10


mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10

mongoc                       mongoc                       mongoc
               EBS – RAID1                 EBS – RAID1                  EBS – RAID1
Sharded with Disaster Recovery Site
                         Region 1                                     Region 2

  Availability Zone A               Availability Zone B          Availability Zone A

mongod                          mongod                         mongod
                                                               hidden
              EBS – RAID10                      EBS – RAID10                 EBS – RAID10


mongod                          mongod                         mongod
                                                               hidden
              EBS – RAID10                      EBS – RAID10                 EBS – RAID10


mongod                          mongod                         mongod
                                                               hidden
              EBS – RAID10                      EBS – RAID10                 EBS – RAID10

mongoc                          mongoc                         mongoc
               EBS – RAID1                      EBS – RAID1                  EBS – RAID1
Multi Region Sharded
       Region 1                     Region 2                     Region 3

  Availability Zone A          Availability Zone A          Availability Zone A

mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10


mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10


mongod                       mongod                       mongod
              EBS – RAID10                 EBS – RAID10                 EBS – RAID10

mongoc                       mongoc                       mongoc
              EBS – RAID1                  EBS – RAID1                  EBS – RAID1
SECURITY
Security Groups
Component                          Default Ports
MongoD                             27017, 28017 (Status page)
MongoD Shard Server                27018
MongoS                             27017
ConfigDB                           27019


•    All nodes listen on all interfaces unless –bind_ip is specified
•    Specify alternate port with –port
•    Don’t mix names & IP’s. Names preferred (not ec2s)
•    Use security groups as access control lists
    • (e.g. allow 27017 from app servers, but not everybody)
Provisioning
                              Region 1                                         Region 2

  Group: app-servers                           Group: mongodb                              2.2.2.2




                                                                                 1.1.1.1




$ ec2-create-group mongodb --description "security group for mongodb nodes”
$ ec2-create-group app-servers --description "security group for app servers

## Allow mongodb nodes to talk to each other
$ ec2-authorize mongodb –o mongodb –u <your ec2 account id>

## Allow app servers to talk to mongodb nodes
$ ec2-authorize mongodb –o app-servers –u <your ec2 account id>

## Allow SSH access to MongoDB nodes
$ ec2-authorize mongodb –p 22

## Allow remote MongoDB nodes to talk to cluster
$ ec2-authorize mongodb –s 1.1.1.1 –p 27017
$ ec2-authorize mongodb –s 2.2.2.2 –p 27017
Questions?
• I am
  – Jared Rosoff (jsr@10gen.com)


• Try it out!
  – http://www.mongodb.org/downloads
  – http://www.mongodb.org/display/DOCS/Amazon
    +EC2

Contenu connexe

Similaire à Mongosv 2011 - MongoDB on Amazon EC2

MongoDB in the Cloud -- Mongo Boulder
MongoDB in the Cloud -- Mongo BoulderMongoDB in the Cloud -- Mongo Boulder
MongoDB in the Cloud -- Mongo BoulderJustin Smestad
 
Rocking mongo db on the cloud
Rocking mongo db on the cloudRocking mongo db on the cloud
Rocking mongo db on the cloudMongoDB
 
Data Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsData Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsTony Pearson
 
Data Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsData Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsTony Pearson
 
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Lindsey Aitchison
 
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...Creating Highly Available MongoDB Microservices with Docker Containers and Ku...
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...MongoDB
 
MongoDB and Amazon Web Services: Storage Options for MongoDB Deployments
MongoDB and Amazon Web Services: Storage Options for MongoDB DeploymentsMongoDB and Amazon Web Services: Storage Options for MongoDB Deployments
MongoDB and Amazon Web Services: Storage Options for MongoDB DeploymentsMongoDB
 
Windows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined StorageWindows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined StorageAidan Finn
 
What to know about Amazon Elastic Block Store (EBS)
What to know about Amazon Elastic Block Store (EBS)What to know about Amazon Elastic Block Store (EBS)
What to know about Amazon Elastic Block Store (EBS)LCloud
 
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Amazon Web Services
 
RADOS for Eucalyptus
RADOS for EucalyptusRADOS for Eucalyptus
RADOS for EucalyptusTakuya ASADA
 

Similaire à Mongosv 2011 - MongoDB on Amazon EC2 (11)

MongoDB in the Cloud -- Mongo Boulder
MongoDB in the Cloud -- Mongo BoulderMongoDB in the Cloud -- Mongo Boulder
MongoDB in the Cloud -- Mongo Boulder
 
Rocking mongo db on the cloud
Rocking mongo db on the cloudRocking mongo db on the cloud
Rocking mongo db on the cloud
 
Data Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsData Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage Options
 
Data Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage OptionsData Footprint Reduction: Understanding IBM Storage Options
Data Footprint Reduction: Understanding IBM Storage Options
 
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
Building an Oracle Grid with Oracle VM on Dell Blade Servers and EqualLogic i...
 
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...Creating Highly Available MongoDB Microservices with Docker Containers and Ku...
Creating Highly Available MongoDB Microservices with Docker Containers and Ku...
 
MongoDB and Amazon Web Services: Storage Options for MongoDB Deployments
MongoDB and Amazon Web Services: Storage Options for MongoDB DeploymentsMongoDB and Amazon Web Services: Storage Options for MongoDB Deployments
MongoDB and Amazon Web Services: Storage Options for MongoDB Deployments
 
Windows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined StorageWindows Server 2012 R2 Software-Defined Storage
Windows Server 2012 R2 Software-Defined Storage
 
What to know about Amazon Elastic Block Store (EBS)
What to know about Amazon Elastic Block Store (EBS)What to know about Amazon Elastic Block Store (EBS)
What to know about Amazon Elastic Block Store (EBS)
 
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS ...
 
RADOS for Eucalyptus
RADOS for EucalyptusRADOS for Eucalyptus
RADOS for Eucalyptus
 

Plus de Jared Rosoff

MongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesMongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesJared Rosoff
 
Mongosv 2011 - Sharding
Mongosv 2011 - ShardingMongosv 2011 - Sharding
Mongosv 2011 - ShardingJared Rosoff
 
Mongosv 2011 - Replication
Mongosv 2011 - ReplicationMongosv 2011 - Replication
Mongosv 2011 - ReplicationJared Rosoff
 
Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Jared Rosoff
 
MongoDB on EC2 and EBS
MongoDB on EC2 and EBSMongoDB on EC2 and EBS
MongoDB on EC2 and EBSJared Rosoff
 
Indexing & query optimization
Indexing & query optimizationIndexing & query optimization
Indexing & query optimizationJared Rosoff
 
Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Jared Rosoff
 
Scalable Event Analytics with MongoDB & Ruby on Rails
Scalable Event Analytics with MongoDB & Ruby on RailsScalable Event Analytics with MongoDB & Ruby on Rails
Scalable Event Analytics with MongoDB & Ruby on RailsJared Rosoff
 

Plus de Jared Rosoff (8)

MongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesMongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - Inboxes
 
Mongosv 2011 - Sharding
Mongosv 2011 - ShardingMongosv 2011 - Sharding
Mongosv 2011 - Sharding
 
Mongosv 2011 - Replication
Mongosv 2011 - ReplicationMongosv 2011 - Replication
Mongosv 2011 - Replication
 
Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011
 
MongoDB on EC2 and EBS
MongoDB on EC2 and EBSMongoDB on EC2 and EBS
MongoDB on EC2 and EBS
 
Indexing & query optimization
Indexing & query optimizationIndexing & query optimization
Indexing & query optimization
 
Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010
 
Scalable Event Analytics with MongoDB & Ruby on Rails
Scalable Event Analytics with MongoDB & Ruby on RailsScalable Event Analytics with MongoDB & Ruby on Rails
Scalable Event Analytics with MongoDB & Ruby on Rails
 

Dernier

Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfOverkill Security
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Angeliki Cooney
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc
 

Dernier (20)

Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 

Mongosv 2011 - MongoDB on Amazon EC2

  • 1. Amazon Web Services Jared Rosoff @forjared
  • 2. Agenda • Single node – Instance sizing – Storage configuration – OS configuration • Replica Set – Availability Zones – Disaster Recovery – Multi Data-Center • Sharded Cluster
  • 3. INSTANCES, STORAGE AND SINGLE NODE CONFIGURATION
  • 4. Instance Sizing 32-bit = Don’t Use Small Arbiter / Config Big 32-bit = Don’t Use Huge
  • 5. OS • Amazon OS now an option • Turn off atime • Raise file descriptor limits cat >> /etc/security/limits.conf << EOF * hard nofile 65536 * soft nofile 65536 EOF • DO NOT use large VM pages • Use ext4, xfs • Use RAID – RAID10 on MongoD – RAID1 on ConfigbDB • Warning! Known problems with Ubuntu 10.04 & EBS – https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853 – https://bugzilla.kernel.org/show_bug.cgi?id=16991
  • 6. A MongoDB Data Node This is your typical database node. Used for actual data storage. We’ll use this icon to represent this type of node throughout this prezo. • 64-bit instance mongod • More RAM == better • Run EXT4 or XFS file system • Turn off atime & diratime EBS – RAID10 • EBS volumes in RAID10
  • 7. A MongoDB Config Server Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements. • 64-bit instance – Micro is fine mongoc • EBS volumes in RAID1 EBS – RAID1
  • 8. A MongoDB Arbiter Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements. • Micro is fine mongod -- • No storage requirements arbiter • Must be separate node from rest of replica set
  • 10. Single Region Replica Set Region Availability Zone A Availability Zone B Availability Zone C mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10
  • 11. Disaster Recovery Site Region 1 Region 2 Availability Zone A Availability Zone B Availability Zone A mongod mongod mongod Hidden EBS – RAID10 EBS – RAID10 EBS – RAID10
  • 12. Multi Data Center Region 1 Region 2 Region 3 Availability Zone A Availability Zone A Availability Zone A mongod mongod mongod Priority 1 EBS – RAID10 EBS – RAID10 EBS – RAID10
  • 14. Single Region Sharded Cluster Region Availability Zone A Availability Zone B Availability Zone C mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
  • 15. Sharded with Disaster Recovery Site Region 1 Region 2 Availability Zone A Availability Zone B Availability Zone A mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod hidden EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
  • 16. Multi Region Sharded Region 1 Region 2 Region 3 Availability Zone A Availability Zone A Availability Zone A mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongod mongod mongod EBS – RAID10 EBS – RAID10 EBS – RAID10 mongoc mongoc mongoc EBS – RAID1 EBS – RAID1 EBS – RAID1
  • 18. Security Groups Component Default Ports MongoD 27017, 28017 (Status page) MongoD Shard Server 27018 MongoS 27017 ConfigDB 27019 • All nodes listen on all interfaces unless –bind_ip is specified • Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s) • Use security groups as access control lists • (e.g. allow 27017 from app servers, but not everybody)
  • 19. Provisioning Region 1 Region 2 Group: app-servers Group: mongodb 2.2.2.2 1.1.1.1 $ ec2-create-group mongodb --description "security group for mongodb nodes” $ ec2-create-group app-servers --description "security group for app servers ## Allow mongodb nodes to talk to each other $ ec2-authorize mongodb –o mongodb –u <your ec2 account id> ## Allow app servers to talk to mongodb nodes $ ec2-authorize mongodb –o app-servers –u <your ec2 account id> ## Allow SSH access to MongoDB nodes $ ec2-authorize mongodb –p 22 ## Allow remote MongoDB nodes to talk to cluster $ ec2-authorize mongodb –s 1.1.1.1 –p 27017 $ ec2-authorize mongodb –s 2.2.2.2 –p 27017
  • 20. Questions? • I am – Jared Rosoff (jsr@10gen.com) • Try it out! – http://www.mongodb.org/downloads – http://www.mongodb.org/display/DOCS/Amazon +EC2

Notes de l'éditeur

  1. Why RAID 10? EBS failures don’t take down the node. Why EBS? More IOPs Easy replacement of volumes Easy to take backups
  2. Why RAID1?Driver failure is a non issue Why EBS? Easy to take snapshots
  3. Use priorities if you want to prefer one region as master over another. Can be extended to more than 3 regions Can have more than one node in each region