SlideShare une entreprise Scribd logo
1  sur  40
Télécharger pour lire hors ligne
MongoDB
      at
Community Engine
About me

Lead platform engineer
mathieu.kempe@communityengine.com
@mathieukempe
Agenda

Brain dump on our experience with MongoDB
• Why NoSQL
• Why we chose MongoDB
• Moving away from an hybrid solution
• MongoDB and Amazon
• SOLR and MongoDB
• Ease of development
• Zero downtime database deployment
About Community Engine
• Social network based on locality and
  small business
• Launching in April
• Built on:
    • ASP.NET MVC 3
    • Amazon Web services
    • MongoDB
    • SOLR
    • Mahout
    • …
NoSQL?

• How to store the big amounts of data required
  in social networking applications
• Data complexity, NoSQL handle hierarchical
  and graph data structures better
• Change management is always difficult with
  RDBMS
• Scaling
Why we chose MongoDB?

• Reviewed different products RavenDB, CouchDB...
• Selected MongoDB because we had the best
  experience
   – Very easy to install and get started
   – Great developer experience
   – Replication very easy to setup
   – Good documentation
   – Much of the convenience of SQL, Dynamic
     Queries, Indexing
MongoDB at community engine
Why Hybrid?

• Team had a lot of experience with SQL Server
  and Entity Framework
• Reporting
• Transaction
MongoDB at community engine
No more SQL Server
• Simplify our infrastructure
• Easier to Backup
• Better performance, not slowed down by SQL
  Server, too many queries joined in the
  application
• Development speed
• Lower cost
Transaction?

Transactions we could go around that using the atomic
  document updates and a good schema design

MongoDB supports atomic operations on single
 document.

When transactions across documents are needed
Two phase commits
Hosting with Amazon Web Service

• Elasticity and scalability
• Configure MongoDB using Amazon EC2 instance
  bundled into an AMI.
• 64 bits EC2 instance
• Raid10 + EBS volumes
• Multi-datacenter 3-node replica set in different
  availability zone
• Use secondaries for zero downtime backup
• We are not yet using sharded replica sets
MongoDB at community engine
What about durability?


Use journaling
Use replica sets
Critical writes

Verify that replication is working at write time

mongodb://host1,host2,host3/?safe=true;w=2;

•safe=true : Use safemode
•w=2: wmode, connect to a replica set waiting for
replication to succeed on the majority of nodes
MongoDB at community engine
MongoDB at community engine
MongoDB at community engine
MongoDB at community engine
MongoDB at community engine
Why we kept SOLR?

•   Right tool for the right job
•   Proven technology
•   SOLR best solution for Full Text Indexing
•   Faceted search, Spelling suggestions…
•   Team already skilled with SOLR
•   SOLR scales well
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
MongoDB/SOLR How we do it
Ease of development
Hierarchical data in SQL Server
Single table
Mongo Database Schema

Using Type discriminator

{
     "_id" : ObjectId("4f504e7acd3e1c190ce04198"),
     "_t" : "PhotoSpark",
     "Photo" : "MyMotorcycle.png "
    "DateCreated" : ISODate("2012-02-24T09:23:12.246Z")
}
{
     "_id" : ObjectId("4f504e7ccd3e1c190ce04199"),
     "_t" : "PostSpark",
     "Body" : "Hello World“
    " DateCreated" : ISODate("2012-02-28T10:44:12.858Z")
}
MongoDB at community engine
MongoDB at community engine
MongoDB at community engine
Views
                      


                                         




                                         




                
Deployment of database with zero downtime

• We release every week
• We aim at zero downtime
• Our domain model change often
Deployment of database with zero downtime

Make sure that our code can handle both
 "versions" of the data structure
When saving we updates to the new structure
Deployment of database with zero downtime

• Use a migration script
Deployment of database with zero downtime

  Expansion Script   Deploy new version   Compression Script




                                                               t
Questions?
Thank you!

mathieu.kempe@communityengine.com
@mathieukempe

Contenu connexe

Tendances

Docker y azure container service
Docker y azure container serviceDocker y azure container service
Docker y azure container serviceFernando Mejía
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Oren Eini
 
Application of nodejs in epsilon mobile
Application of nodejs in epsilon mobileApplication of nodejs in epsilon mobile
Application of nodejs in epsilon mobileTony Vo
 
Cloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.comCloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.comAWS Vietnam Community
 
Containerization - The DevOps Revolution
Containerization - The DevOps RevolutionContainerization - The DevOps Revolution
Containerization - The DevOps RevolutionYulian Slobodyan
 
Aws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and DevelopersAws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and DevelopersDylan Burris
 
Cnam cours azure web sites
Cnam cours azure web sitesCnam cours azure web sites
Cnam cours azure web sitesAymeric Weinbach
 
Running MongoDB in the Cloud
Running MongoDB in the CloudRunning MongoDB in the Cloud
Running MongoDB in the CloudTony Tam
 
Building Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics worldBuilding Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics worldOren Eini
 
How Microsoft learned to love Java
How Microsoft learned to love JavaHow Microsoft learned to love Java
How Microsoft learned to love JavaBrian Benz
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Siteozzyr
 
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...Marina Peregud
 
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWSCustomer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWSAmazon Web Services
 
Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda Outlyer
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPressDylan Burris
 
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup LeipzigVisualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzigroot360 GmbH
 
Azure virtual machine-network
Azure virtual machine-networkAzure virtual machine-network
Azure virtual machine-networkThi Nguyen Dinh
 

Tendances (19)

Docker y azure container service
Docker y azure container serviceDocker y azure container service
Docker y azure container service
 
Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)Know thy cost (or where performance problems lurk)
Know thy cost (or where performance problems lurk)
 
Architecting on The Cloud
Architecting on The CloudArchitecting on The Cloud
Architecting on The Cloud
 
Application of nodejs in epsilon mobile
Application of nodejs in epsilon mobileApplication of nodejs in epsilon mobile
Application of nodejs in epsilon mobile
 
Cloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.comCloudsolutionday 2016: Docker & FAAS at getvero.com
Cloudsolutionday 2016: Docker & FAAS at getvero.com
 
Containerization - The DevOps Revolution
Containerization - The DevOps RevolutionContainerization - The DevOps Revolution
Containerization - The DevOps Revolution
 
Aws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and DevelopersAws 12 Month Free Tier for Web Designers and Developers
Aws 12 Month Free Tier for Web Designers and Developers
 
Cnam cours azure web sites
Cnam cours azure web sitesCnam cours azure web sites
Cnam cours azure web sites
 
Running MongoDB in the Cloud
Running MongoDB in the CloudRunning MongoDB in the Cloud
Running MongoDB in the Cloud
 
Building Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics worldBuilding Codealike: a journey into the developers analytics world
Building Codealike: a journey into the developers analytics world
 
How Microsoft learned to love Java
How Microsoft learned to love JavaHow Microsoft learned to love Java
How Microsoft learned to love Java
 
Optimizing Your WordPress Site
Optimizing Your WordPress SiteOptimizing Your WordPress Site
Optimizing Your WordPress Site
 
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
Антон Бойко "Разделяй и властвуй — набор практик для построения масштабируемо...
 
How and when to use NoSQL
How and when to use NoSQLHow and when to use NoSQL
How and when to use NoSQL
 
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWSCustomer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
Customer Presentaion-LittleBigPlanet: Taking an Idea to Market Using AWS
 
Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda Matt Chung (Independent) - Serverless application with AWS Lambda
Matt Chung (Independent) - Serverless application with AWS Lambda
 
How to Build High Performance : WordPress
How to Build High Performance : WordPressHow to Build High Performance : WordPress
How to Build High Performance : WordPress
 
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup LeipzigVisualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
Visualization of RDS metrics using AWS CLI and JQuery at AWS Usergroup Leipzig
 
Azure virtual machine-network
Azure virtual machine-networkAzure virtual machine-network
Azure virtual machine-network
 

En vedette

ownCloud, tu nube libre
ownCloud, tu nube libreownCloud, tu nube libre
ownCloud, tu nube libreJorge Cacho
 
Entrevista a Jono Bacon
Entrevista a Jono BaconEntrevista a Jono Bacon
Entrevista a Jono BaconJorge Cacho
 
Smarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo costeSmarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo costeJorge Cacho
 
Fortifica tu Wordpress con Latch
Fortifica tu Wordpress con LatchFortifica tu Wordpress con Latch
Fortifica tu Wordpress con LatchJorge Cacho
 
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)Jorge Cacho
 
Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35Jorge Cacho
 
Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)Jorge Cacho
 
Intro to the MongoDB Community
Intro to the MongoDB CommunityIntro to the MongoDB Community
Intro to the MongoDB Communityfrancescapasha
 
The role of the community manager in your organisation
The role of the community manager in your organisationThe role of the community manager in your organisation
The role of the community manager in your organisationBlaise Grimes-Viort
 

En vedette (9)

ownCloud, tu nube libre
ownCloud, tu nube libreownCloud, tu nube libre
ownCloud, tu nube libre
 
Entrevista a Jono Bacon
Entrevista a Jono BaconEntrevista a Jono Bacon
Entrevista a Jono Bacon
 
Smarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo costeSmarttop: un servidor web (y mucho más) de bajo coste
Smarttop: un servidor web (y mucho más) de bajo coste
 
Fortifica tu Wordpress con Latch
Fortifica tu Wordpress con LatchFortifica tu Wordpress con Latch
Fortifica tu Wordpress con Latch
 
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
Raspberry Pi: tutoriales (servidor web, ownCloud y XBMC)
 
Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35Raspberry Pi: un ordenador por $35
Raspberry Pi: un ordenador por $35
 
Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)Tiny Tiny RSS(bye bye Google Reader)
Tiny Tiny RSS(bye bye Google Reader)
 
Intro to the MongoDB Community
Intro to the MongoDB CommunityIntro to the MongoDB Community
Intro to the MongoDB Community
 
The role of the community manager in your organisation
The role of the community manager in your organisationThe role of the community manager in your organisation
The role of the community manager in your organisation
 

Similaire à MongoDB at community engine

MongoDB.pptx
MongoDB.pptxMongoDB.pptx
MongoDB.pptxSigit52
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB
 
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)Shrey Batra
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDBFoundationDB
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaHelen Rogers
 
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft ShopThe Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft ShopJeff Smoley
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Emerson Eduardo Rodrigues Von Staffen
 
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...Amazon Web Services
 
Introduction to MERN Stack
Introduction to MERN StackIntroduction to MERN Stack
Introduction to MERN StackSurya937648
 
web development with mern stack in power point
web development with mern stack in power pointweb development with mern stack in power point
web development with mern stack in power pointRAMKUMARRIT20
 
Final year presentation topicssssss in 1
Final year presentation topicssssss in 1Final year presentation topicssssss in 1
Final year presentation topicssssss in 1RAMKUMARRIT20
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)emiltamas
 
Mongo db first steps with csharp
Mongo db first steps with csharpMongo db first steps with csharp
Mongo db first steps with csharpSerdar Buyuktemiz
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?DATAVERSITY
 
An introduction to Node.js
An introduction to Node.jsAn introduction to Node.js
An introduction to Node.jsKasey McCurdy
 

Similaire à MongoDB at community engine (20)

MongoDB.pptx
MongoDB.pptxMongoDB.pptx
MongoDB.pptx
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
 
Managing Your Cloud Assets
Managing Your Cloud AssetsManaging Your Cloud Assets
Managing Your Cloud Assets
 
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
MongoDB MUG Delhi NCR - December 19 2020 (Cloud Security)
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
 
Microservices
MicroservicesMicroservices
Microservices
 
Aws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon ElishaAws-What You Need to Know_Simon Elisha
Aws-What You Need to Know_Simon Elisha
 
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft ShopThe Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
The Perils and Triumphs of using Cassandra at a .NET/Microsoft Shop
 
20120306 dublin js
20120306 dublin js20120306 dublin js
20120306 dublin js
 
Meanstack overview
Meanstack overviewMeanstack overview
Meanstack overview
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
 
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
 
Windows Azure introduction
Windows Azure introductionWindows Azure introduction
Windows Azure introduction
 
Introduction to MERN Stack
Introduction to MERN StackIntroduction to MERN Stack
Introduction to MERN Stack
 
web development with mern stack in power point
web development with mern stack in power pointweb development with mern stack in power point
web development with mern stack in power point
 
Final year presentation topicssssss in 1
Final year presentation topicssssss in 1Final year presentation topicssssss in 1
Final year presentation topicssssss in 1
 
Scaling with mongo db (with notes)
Scaling with mongo db (with notes)Scaling with mongo db (with notes)
Scaling with mongo db (with notes)
 
Mongo db first steps with csharp
Mongo db first steps with csharpMongo db first steps with csharp
Mongo db first steps with csharp
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
 
An introduction to Node.js
An introduction to Node.jsAn introduction to Node.js
An introduction to Node.js
 

Dernier

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 

Dernier (20)

Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 

MongoDB at community engine

  • 1. MongoDB at Community Engine
  • 2. About me Lead platform engineer mathieu.kempe@communityengine.com @mathieukempe
  • 3. Agenda Brain dump on our experience with MongoDB • Why NoSQL • Why we chose MongoDB • Moving away from an hybrid solution • MongoDB and Amazon • SOLR and MongoDB • Ease of development • Zero downtime database deployment
  • 4. About Community Engine • Social network based on locality and small business • Launching in April • Built on: • ASP.NET MVC 3 • Amazon Web services • MongoDB • SOLR • Mahout • …
  • 5. NoSQL? • How to store the big amounts of data required in social networking applications • Data complexity, NoSQL handle hierarchical and graph data structures better • Change management is always difficult with RDBMS • Scaling
  • 6. Why we chose MongoDB? • Reviewed different products RavenDB, CouchDB... • Selected MongoDB because we had the best experience – Very easy to install and get started – Great developer experience – Replication very easy to setup – Good documentation – Much of the convenience of SQL, Dynamic Queries, Indexing
  • 8. Why Hybrid? • Team had a lot of experience with SQL Server and Entity Framework • Reporting • Transaction
  • 10. No more SQL Server • Simplify our infrastructure • Easier to Backup • Better performance, not slowed down by SQL Server, too many queries joined in the application • Development speed • Lower cost
  • 11. Transaction? Transactions we could go around that using the atomic document updates and a good schema design MongoDB supports atomic operations on single document. When transactions across documents are needed Two phase commits
  • 12. Hosting with Amazon Web Service • Elasticity and scalability • Configure MongoDB using Amazon EC2 instance bundled into an AMI. • 64 bits EC2 instance • Raid10 + EBS volumes • Multi-datacenter 3-node replica set in different availability zone • Use secondaries for zero downtime backup • We are not yet using sharded replica sets
  • 14. What about durability? Use journaling Use replica sets
  • 15. Critical writes Verify that replication is working at write time mongodb://host1,host2,host3/?safe=true;w=2; •safe=true : Use safemode •w=2: wmode, connect to a replica set waiting for replication to succeed on the majority of nodes
  • 21. Why we kept SOLR? • Right tool for the right job • Proven technology • SOLR best solution for Full Text Indexing • Faceted search, Spelling suggestions… • Team already skilled with SOLR • SOLR scales well
  • 28. Hierarchical data in SQL Server
  • 30. Mongo Database Schema Using Type discriminator { "_id" : ObjectId("4f504e7acd3e1c190ce04198"), "_t" : "PhotoSpark", "Photo" : "MyMotorcycle.png " "DateCreated" : ISODate("2012-02-24T09:23:12.246Z") } { "_id" : ObjectId("4f504e7ccd3e1c190ce04199"), "_t" : "PostSpark", "Body" : "Hello World“ " DateCreated" : ISODate("2012-02-28T10:44:12.858Z") }
  • 34. Views                                                                    
  • 35. Deployment of database with zero downtime • We release every week • We aim at zero downtime • Our domain model change often
  • 36. Deployment of database with zero downtime Make sure that our code can handle both "versions" of the data structure When saving we updates to the new structure
  • 37. Deployment of database with zero downtime • Use a migration script
  • 38. Deployment of database with zero downtime Expansion Script Deploy new version Compression Script t

Notes de l'éditeur

  1. Brain dump on why we started to use MongoDB Why we moved away from a solution using MongoDB and SQL Server
  2. Moved to this infra
  3. If the primary crash we have the data in at least one of the secondary Second node has priority greater than or equal to other eligible nodes in the set
  4. Update is different as we do sometime Update in place Update in place are a query and an update in one command