Soumettre la recherche
Mettre en ligne
Postgres Sql at TripAdvisor
•
5 j'aime
•
1,758 vues
Don O'Neill
Suivre
Slide Deck from my recent talk at the first New England PostgreSQL User group meeting.
Lire moins
Lire la suite
Technologie
Signaler
Partager
Signaler
Partager
1 sur 10
Recommandé
BigDataInOperationsV8
BigDataInOperationsV8
Don O'Neill
Managing Large Scale Financial Time-Series Data with Graphs
Managing Large Scale Financial Time-Series Data with Graphs
Objectivity
Spark Summit Keynote by Suren Nathan
Spark Summit Keynote by Suren Nathan
Spark Summit
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
Shawn Jones
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Matt Stubbs
Memrise presentation @ London Snowplow meetup
Memrise presentation @ London Snowplow meetup
idan_by
Treasure Data From MySQL to Redshift
Treasure Data From MySQL to Redshift
Treasure Data, Inc.
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
confluent
Recommandé
BigDataInOperationsV8
BigDataInOperationsV8
Don O'Neill
Managing Large Scale Financial Time-Series Data with Graphs
Managing Large Scale Financial Time-Series Data with Graphs
Objectivity
Spark Summit Keynote by Suren Nathan
Spark Summit Keynote by Suren Nathan
Spark Summit
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
It’s All About The Cards: Sharing on Social Media Encouraged HTML Metadata G...
Shawn Jones
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Big Data LDN 2016: Out of the Data Warehouses, and into the Data Lakes and St...
Matt Stubbs
Memrise presentation @ London Snowplow meetup
Memrise presentation @ London Snowplow meetup
idan_by
Treasure Data From MySQL to Redshift
Treasure Data From MySQL to Redshift
Treasure Data, Inc.
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
Simplifying Event Streaming: Tools for Location Transparency and Data Evoluti...
confluent
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
DataWorks Summit/Hadoop Summit
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Streamsets Inc.
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Spark Summit
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Nascenia IT
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
✔ Eric David Benari, PMP
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Natalino Busa
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Spark Summit
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
dfilppi
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
SAP Technology
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Databricks
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
Marc Fielding
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
DataStax
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Spark Summit
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
alistair_hann
Presto summit israel 2019-04
Presto summit israel 2019-04
Ori Reshef
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Spark Summit
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
SingleStore
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Ori Reshef
Senior database administrator
Senior database administrator
Mustafa EL-Masry
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
Contenu connexe
Tendances
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
DataWorks Summit/Hadoop Summit
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Streamsets Inc.
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Spark Summit
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Nascenia IT
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
✔ Eric David Benari, PMP
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Natalino Busa
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Spark Summit
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
dfilppi
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
SAP Technology
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Databricks
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
Marc Fielding
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
DataStax
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Spark Summit
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
alistair_hann
Presto summit israel 2019-04
Presto summit israel 2019-04
Ori Reshef
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Spark Summit
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
SingleStore
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Ori Reshef
Tendances
(20)
The Stream is the Database - Revolutionizing Healthcare Data Architecture
The Stream is the Database - Revolutionizing Healthcare Data Architecture
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Enabling Next Gen Analytics with Azure Data Lake and StreamSets
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Virtualizing Analytics with Apache Spark: Keynote by Arsalan Tavakoli
Spark Summit Keynote by Seshu Adunuthula
Spark Summit Keynote by Seshu Adunuthula
Introduction to basic data analytics tools
Introduction to basic data analytics tools
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Database Camp 2016 @ United Nations, NYC - Michael Glukhovsky, Co-Founder, Re...
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Spark and the Enterprise by Tony Baer
Spark and the Enterprise by Tony Baer
Middle Tier Scalability - Present and Future
Middle Tier Scalability - Present and Future
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
MongoDB World 2019: Streaming ETL on the Shoulders of Giants
Spark Usage in Enterprise Business Operations
Spark Usage in Enterprise Business Operations
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Moving eBay’s Data Warehouse Over to Apache Spark – Spark as Core ETL Platfor...
Cassandra Essentials Day Cambridge
Cassandra Essentials Day Cambridge
DataStax Enterprise in Practice (Field Notes)
DataStax Enterprise in Practice (Field Notes)
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
SomeSQL at Skyscanner - Scaling in a changing world of databases and hardware
Presto summit israel 2019-04
Presto summit israel 2019-04
Spark Summit presentation by Ken Tsai
Spark Summit presentation by Ken Tsai
Winning the On-Demand Economy with Spark and Predictive Analytics
Winning the On-Demand Economy with Spark and Predictive Analytics
Data analytics at a petabyte scale final
Data analytics at a petabyte scale final
Similaire à Postgres Sql at TripAdvisor
Senior database administrator
Senior database administrator
Mustafa EL-Masry
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Directi Group
Full Stack Web Developer (MERN STACK Developer.pptx
Full Stack Web Developer (MERN STACK Developer.pptx
RamudgarYadav
Top local databases for react native app development
Top local databases for react native app development
SameerShaik43
Neethu_Abraham
Neethu_Abraham
Neethu Abraham
List of Top Local Databases used for react native app developement in 2022
List of Top Local Databases used for react native app developement in 2022
Shelly Megan
Introduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
Cecile Le Pape
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Cécile Poyet
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Hortonworks
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Cécile Poyet
Metadata describes about data
Metadata describes about data
Dev Sanskriti Vishwavidyalaya (University)
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
confluent
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
Dynatrace
Whats new in Oracle Database 12c release 12.1.0.2
Whats new in Oracle Database 12c release 12.1.0.2
Connor McDonald
Introduction to Cloud Service Design
Introduction to Cloud Service Design
evancmiller
Cloud-Native Data: What data questions to ask when building cloud-native apps
Cloud-Native Data: What data questions to ask when building cloud-native apps
VMware Tanzu
React Native Local Database For Super Smart Applications.pdf
React Native Local Database For Super Smart Applications.pdf
Katy Slemon
Rahul_Resume_310116
Rahul_Resume_310116
RAHUL MEHROTRA
Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101
MongoDB
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar Slides
Hortonworks
Similaire à Postgres Sql at TripAdvisor
(20)
Senior database administrator
Senior database administrator
Handling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
Full Stack Web Developer (MERN STACK Developer.pptx
Full Stack Web Developer (MERN STACK Developer.pptx
Top local databases for react native app development
Top local databases for react native app development
Neethu_Abraham
Neethu_Abraham
List of Top Local Databases used for react native app developement in 2022
List of Top Local Databases used for react native app developement in 2022
Introduction to NoSQL and Couchbase
Introduction to NoSQL and Couchbase
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Boost Performance with Scala – Learn From Those Who’ve Done It!
Metadata describes about data
Metadata describes about data
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Achieve Sub-Second Analytics on Apache Kafka with Confluent and Imply
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
Whats new in Oracle Database 12c release 12.1.0.2
Whats new in Oracle Database 12c release 12.1.0.2
Introduction to Cloud Service Design
Introduction to Cloud Service Design
Cloud-Native Data: What data questions to ask when building cloud-native apps
Cloud-Native Data: What data questions to ask when building cloud-native apps
React Native Local Database For Super Smart Applications.pdf
React Native Local Database For Super Smart Applications.pdf
Rahul_Resume_310116
Rahul_Resume_310116
Ops Jumpstart: MongoDB Administration 101
Ops Jumpstart: MongoDB Administration 101
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar Slides
Dernier
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Neo4j
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
BookNet Canada
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
Pooja Nehwal
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Safe Software
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Delhi Call girls
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
ThousandEyes
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Ridwan Fadjar
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Alan Dix
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
Allon Mureinik
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Pixlogix Infotech
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Memoori
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
XfilesPro
Dernier
(20)
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Pigging Solutions Piggable Sweeping Elbows
Pigging Solutions Piggable Sweeping Elbows
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
Postgres Sql at TripAdvisor
1.
No, we’re not
a travel agency www.tripadvisor.com/careers
2.
Postgres at TripAdvisor
By Don O’Neill & Boris Partensky 2 © 2011 TripAdvisor LLC All rights reserved.
3.
TripAdvisor is the
world’s largesttravel site 3 © 2011 TripAdvisor LLC All rights reserved.
4.
4 40M monthly
visitors 4M visitors/day 20+M registered members 45+M reviews & opinions © 2011 TripAdvisor LLC All rights reserved.
5.
Business runs on
Postgres Large Mission critical deployment Combination of 8.3 and 8.4 Site Technologies include Linux (CentOS), Apache Java, Tomcat, Velocity, and Memcached No commercial Software – All Open Source Web Scale Deployment Over 100 Frontend and Backend servers per Data Center 14 Postgres Database Servers per Data Center DR Data Center with Replication between the two Platform 5 © 2011 TripAdvisor LLC All rights reserved.
6.
33 Postgres Servers
across 3 Data Centers Not including Dev and QA servers SSD’s instead of Spinning disks for select servers Multiple Read Only DB Servers Load Balanced using BigIP Sync’d using DB Mirror replication Just under 1 TB of data in multiple databases Sharded Pin Table Over 1.5 billion rows Sharded to improve read and write performance Spread over 12 databases on 2 Servers Scale out to multiple servers as needed Routing logic built into our application layer By the Numbers 6 © 2011 TripAdvisor LLC All rights reserved.
7.
All Servers configured
as Active/Passive Cluster Not using Shared Disks or SAN’s DRBD for Block level replication between systems Heartbeat for automatic failover between nodes Data Replicated to DR Site TripAdvisor has 2 Data centers (MA and VA) One site “Live” at a time Equal capacity at both sites Routinely switch between sites Data Replicated between data centers Redundancy/Failover 7 © 2011 TripAdvisor LLC All rights reserved.
8.
Replication based on
DBmirror Like Slonyonly deprecated Community project once part of the core distribution Trigger based Master/Slave Replication Written in C and Perl All replicated tables must have primary keys TripAdvisor Updates include A Rewrite in Java! Support for SQL Server Lots of Hardening for large scale use Replication 8 © 2011 TripAdvisor LLC All rights reserved.
9.
20 TB Data
Warehouse today Expecting to grow significantly in 12 months 16 Node cluster expanding to 24 Based on Hadoop and Hive Reporting and Cubes built with SQL Server End users access the data via Excel and Reporting Services Critical for the growth of the business Long term data storage in Hadoop and not our SAN! Mobile Applications Award winning travel application Supported in iPhone, iPad, Android, Nokia, Windows, Palm http://www.tripadvisor.com/MobileApps Facebook Integration More and more users logging in via Facebook Better social experience on the site Major Initiatives 9 © 2011 TripAdvisor LLC All rights reserved.
10.
Q&A Questions? Did
I mention we’re hiring? 10 © 2011 TripAdvisor LLC All rights reserved.