SlideShare une entreprise Scribd logo
1  sur  14
Clojure Performance

Measuring throughput
and latency
Of a news feed
Web service
Written in Clojure
Features
●

Participant Profiles

●

Social Graph

●

Social Broadcast

●

Inbound Activity

●

Outbound Activity

●

Keyword Search
Components
●

Postgresql

●

Cassandra

●

Solr

●

Kafka

●

Redis

●

Jetty
Source Code
●

https://github.com/gengstrand/clojure-newsfeed
AWS Deployment
●

Ubuntu Server 12.04.3 LTS 64 bit

●

Servers (no provisioned IOPs)
–

●

Database (RDS)
–

●

m1.medium (2 ECUs, 1 vCPUs, 3.7 GiB
memory, 1 x 410 GiB Storage Capacity)
db.m1.medium

Client Load Machine
–

m1.large (4 ECUs, 2 vCPUs, 7.5 GiB
memory, 2 x 420 GiB Storage Capacity)
Throughput
Throughput
●

100 threads

●

Connection pool max size: 50

●

5 < Social Broadcast < 50

●

10 posts per user

●

10% searches

●

55 posts per second
Latency Overall
●

Poor at the beginning – cold cache

●

Really good 15 minutes in

●

Bad spike near the end of the first hour

●

Steady performance in the second hour
Postgresql Latency
Cassandra Latency
Solr Latency
Web Service Latency
Summary
●

Cloud Spike

●

Postgresql latency similar to Cassandra

●

Solr indexing twice as slow

●

Between 50 and 70 requests per second

●

Clojure just as fast as Java

●

Jetty performed well
Thanks

Glenn Engstrand

Contenu connexe

Tendances

Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
OdessaJS Conf
 

Tendances (20)

Scaling Islandora
Scaling IslandoraScaling Islandora
Scaling Islandora
 
Security Monitoring for big Infrastructures without a Million Dollar budget
Security Monitoring for big Infrastructures without a Million Dollar budgetSecurity Monitoring for big Infrastructures without a Million Dollar budget
Security Monitoring for big Infrastructures without a Million Dollar budget
 
X3
X3X3
X3
 
Eko10 workshop - OPEN SOURCE DATABASE MONITORING
Eko10 workshop - OPEN SOURCE DATABASE MONITORINGEko10 workshop - OPEN SOURCE DATABASE MONITORING
Eko10 workshop - OPEN SOURCE DATABASE MONITORING
 
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''
 
Ruby e xmpp
Ruby e xmppRuby e xmpp
Ruby e xmpp
 
Solving some of the scalability problems at booking.com
Solving some of the scalability problems at booking.comSolving some of the scalability problems at booking.com
Solving some of the scalability problems at booking.com
 
Systems management - UltraESB
Systems management - UltraESBSystems management - UltraESB
Systems management - UltraESB
 
Breaking the RpiDocker challenge
Breaking the RpiDocker challenge Breaking the RpiDocker challenge
Breaking the RpiDocker challenge
 
Writing a fast HTTP parser
Writing a fast HTTP parserWriting a fast HTTP parser
Writing a fast HTTP parser
 
Distributed scheduler hell (MicroXChg 2017 Berlin)
Distributed scheduler hell (MicroXChg 2017 Berlin)Distributed scheduler hell (MicroXChg 2017 Berlin)
Distributed scheduler hell (MicroXChg 2017 Berlin)
 
RGW S3: Features vs deep compatibility - Robin Johnson
RGW S3: Features vs deep compatibility  - Robin JohnsonRGW S3: Features vs deep compatibility  - Robin Johnson
RGW S3: Features vs deep compatibility - Robin Johnson
 
Kafka Summit SF 2017 - Infrastructure for Streaming Applications
Kafka Summit SF 2017 - Infrastructure for Streaming Applications Kafka Summit SF 2017 - Infrastructure for Streaming Applications
Kafka Summit SF 2017 - Infrastructure for Streaming Applications
 
GEO mapbox geo_api_develop2 Intro
 GEO mapbox geo_api_develop2 Intro GEO mapbox geo_api_develop2 Intro
GEO mapbox geo_api_develop2 Intro
 
Machinekit - The Python Machinetalk Bindings
Machinekit - The Python Machinetalk BindingsMachinekit - The Python Machinetalk Bindings
Machinekit - The Python Machinetalk Bindings
 
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
My Learnings on Setting up a Kubernetes Cluster on AWS using Kubernetes Opera...
 
FaaS you like it (if Shakespeare had written Functions-as-a-Service)
FaaS you like it (if Shakespeare had written Functions-as-a-Service)FaaS you like it (if Shakespeare had written Functions-as-a-Service)
FaaS you like it (if Shakespeare had written Functions-as-a-Service)
 
Google file system
Google file systemGoogle file system
Google file system
 
Timur Shemsedinov "Эволюция архитектуры ИС"
Timur Shemsedinov "Эволюция архитектуры ИС"Timur Shemsedinov "Эволюция архитектуры ИС"
Timur Shemsedinov "Эволюция архитектуры ИС"
 
ng-docs
ng-docsng-docs
ng-docs
 

En vedette

En vedette (7)

Gast College
Gast CollegeGast College
Gast College
 
Drupaljam 8 zoetermeer opening
Drupaljam 8 zoetermeer openingDrupaljam 8 zoetermeer opening
Drupaljam 8 zoetermeer opening
 
Opportunity Eye for Bloggers
Opportunity Eye for BloggersOpportunity Eye for Bloggers
Opportunity Eye for Bloggers
 
Drupaljam 2013 rotterdam opening
Drupaljam 2013 rotterdam openingDrupaljam 2013 rotterdam opening
Drupaljam 2013 rotterdam opening
 
7 Reasons Why Cost Conscious Companies are Revisiting Smart Outsourcing
7 Reasons Why Cost Conscious Companies are Revisiting Smart Outsourcing7 Reasons Why Cost Conscious Companies are Revisiting Smart Outsourcing
7 Reasons Why Cost Conscious Companies are Revisiting Smart Outsourcing
 
Drupal Ncrv
Drupal NcrvDrupal Ncrv
Drupal Ncrv
 
Presentatie sponsoring en opensource conferenties FOSS11
Presentatie sponsoring en opensource conferenties FOSS11Presentatie sponsoring en opensource conferenties FOSS11
Presentatie sponsoring en opensource conferenties FOSS11
 

Similaire à Clojure News Feed Performance Testing

Similaire à Clojure News Feed Performance Testing (20)

Event driven architectures with Kinesis
Event driven architectures with KinesisEvent driven architectures with Kinesis
Event driven architectures with Kinesis
 
Logs @ OVHcloud
Logs @ OVHcloudLogs @ OVHcloud
Logs @ OVHcloud
 
USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a Month
USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a MonthUSENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a Month
USENIX LISA15: How TubeMogul Handles over One Trillion HTTP Requests a Month
 
FastNetMon and Metrics
FastNetMon and MetricsFastNetMon and Metrics
FastNetMon and Metrics
 
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
Kubernetes @ Squarespace (SRE Portland Meetup October 2017)
 
Cloud arch patterns
Cloud arch patternsCloud arch patterns
Cloud arch patterns
 
Debugging data pipelines @OLA by Karan Kumar
Debugging data pipelines @OLA by Karan KumarDebugging data pipelines @OLA by Karan Kumar
Debugging data pipelines @OLA by Karan Kumar
 
Netflix Data Pipeline With Kafka
Netflix Data Pipeline With KafkaNetflix Data Pipeline With Kafka
Netflix Data Pipeline With Kafka
 
Netflix Data Pipeline With Kafka
Netflix Data Pipeline With KafkaNetflix Data Pipeline With Kafka
Netflix Data Pipeline With Kafka
 
Comparison between zookeeper, etcd 3 and other distributed coordination systems
Comparison between zookeeper, etcd 3 and other distributed coordination systemsComparison between zookeeper, etcd 3 and other distributed coordination systems
Comparison between zookeeper, etcd 3 and other distributed coordination systems
 
Performance challenges in software networking
Performance challenges in software networkingPerformance challenges in software networking
Performance challenges in software networking
 
Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...
Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...
Protecting the Web at a scale using consul and Elk / Valentin Chernozemski (S...
 
Ippevent : openshift Introduction
Ippevent : openshift IntroductionIppevent : openshift Introduction
Ippevent : openshift Introduction
 
PGConf.ASIA 2019 Bali - PostgreSQL on K8S at Zalando - Alexander Kukushkin
PGConf.ASIA 2019 Bali - PostgreSQL on K8S at Zalando - Alexander KukushkinPGConf.ASIA 2019 Bali - PostgreSQL on K8S at Zalando - Alexander Kukushkin
PGConf.ASIA 2019 Bali - PostgreSQL on K8S at Zalando - Alexander Kukushkin
 
Netflix Keystone Pipeline at Big Data Bootcamp, Santa Clara, Nov 2015
Netflix Keystone Pipeline at Big Data Bootcamp, Santa Clara, Nov 2015Netflix Keystone Pipeline at Big Data Bootcamp, Santa Clara, Nov 2015
Netflix Keystone Pipeline at Big Data Bootcamp, Santa Clara, Nov 2015
 
Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. - Pulsar Summit SF 2022
Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. - Pulsar Summit SF 2022Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. - Pulsar Summit SF 2022
Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. - Pulsar Summit SF 2022
 
Scaling GraphQL Subscriptions
Scaling GraphQL SubscriptionsScaling GraphQL Subscriptions
Scaling GraphQL Subscriptions
 
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...
 
Netty training
Netty trainingNetty training
Netty training
 
Netty training
Netty trainingNetty training
Netty training
 

Dernier

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Dernier (20)

2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Evaluating the top large language models.pdf
Evaluating the top large language models.pdfEvaluating the top large language models.pdf
Evaluating the top large language models.pdf
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 

Clojure News Feed Performance Testing