Soumettre la recherche
Mettre en ligne
微博cache设计谈
•
Télécharger en tant que KEY, PDF
•
139 j'aime
•
13,703 vues
T
Tim Y
Suivre
Technologie
Signaler
Partager
Signaler
Partager
1 sur 32
Télécharger maintenant
Recommandé
新浪微博Feed服务架构
新浪微博Feed服务架构
XiaoJun Hong
大数据时代feed架构 (ArchSummit Beijing 2014)
大数据时代feed架构 (ArchSummit Beijing 2014)
Tim Y
High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011
Tim Y
微博架构与平台安全
微博架构与平台安全
Tim Y
新浪微博开放平台Redis实战
新浪微博开放平台Redis实战
mysqlops
分布式Key Value Store漫谈
分布式Key Value Store漫谈
Tim Y
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBase
enissoz
[211] HBase 기반 검색 데이터 저장소 (공개용)
[211] HBase 기반 검색 데이터 저장소 (공개용)
NAVER D2
Recommandé
新浪微博Feed服务架构
新浪微博Feed服务架构
XiaoJun Hong
大数据时代feed架构 (ArchSummit Beijing 2014)
大数据时代feed架构 (ArchSummit Beijing 2014)
Tim Y
High Performance Weibo QCon Beijing 2011
High Performance Weibo QCon Beijing 2011
Tim Y
微博架构与平台安全
微博架构与平台安全
Tim Y
新浪微博开放平台Redis实战
新浪微博开放平台Redis实战
mysqlops
分布式Key Value Store漫谈
分布式Key Value Store漫谈
Tim Y
HBase and HDFS: Understanding FileSystem Usage in HBase
HBase and HDFS: Understanding FileSystem Usage in HBase
enissoz
[211] HBase 기반 검색 데이터 저장소 (공개용)
[211] HBase 기반 검색 데이터 저장소 (공개용)
NAVER D2
Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)
Rick Branson
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
涛 吴
Hive+Tez: A performance deep dive
Hive+Tez: A performance deep dive
t3rmin4t0r
[234]멀티테넌트 하둡 클러스터 운영 경험기
[234]멀티테넌트 하둡 클러스터 운영 경험기
NAVER D2
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Noritaka Sekiyama
Introduction to Redis
Introduction to Redis
Dvir Volk
HBase Application Performance Improvement
HBase Application Performance Improvement
Biju Nair
Need for Time series Database
Need for Time series Database
Pramit Choudhary
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
HostedbyConfluent
Hive User Meeting August 2009 Facebook
Hive User Meeting August 2009 Facebook
ragho
ORC File - Optimizing Your Big Data
ORC File - Optimizing Your Big Data
DataWorks Summit
Optimizing Apache Spark SQL Joins
Optimizing Apache Spark SQL Joins
Databricks
Care and Feeding of Catalyst Optimizer
Care and Feeding of Catalyst Optimizer
Databricks
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
NAVER D2
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
DataWorks Summit/Hadoop Summit
MongoDB Fundamentals
MongoDB Fundamentals
MongoDB
HBase Advanced - Lars George
HBase Advanced - Lars George
JAX London
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Databricks
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxData
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010
Tim Y
新浪微博分布式缓存与队列-2013版
新浪微博分布式缓存与队列-2013版
XiaoJun Hong
Contenu connexe
Tendances
Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)
Rick Branson
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
涛 吴
Hive+Tez: A performance deep dive
Hive+Tez: A performance deep dive
t3rmin4t0r
[234]멀티테넌트 하둡 클러스터 운영 경험기
[234]멀티테넌트 하둡 클러스터 운영 경험기
NAVER D2
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Noritaka Sekiyama
Introduction to Redis
Introduction to Redis
Dvir Volk
HBase Application Performance Improvement
HBase Application Performance Improvement
Biju Nair
Need for Time series Database
Need for Time series Database
Pramit Choudhary
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
HostedbyConfluent
Hive User Meeting August 2009 Facebook
Hive User Meeting August 2009 Facebook
ragho
ORC File - Optimizing Your Big Data
ORC File - Optimizing Your Big Data
DataWorks Summit
Optimizing Apache Spark SQL Joins
Optimizing Apache Spark SQL Joins
Databricks
Care and Feeding of Catalyst Optimizer
Care and Feeding of Catalyst Optimizer
Databricks
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
NAVER D2
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
DataWorks Summit/Hadoop Summit
MongoDB Fundamentals
MongoDB Fundamentals
MongoDB
HBase Advanced - Lars George
HBase Advanced - Lars George
JAX London
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Databricks
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxData
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Databricks
Tendances
(20)
Cassandra at Instagram (August 2013)
Cassandra at Instagram (August 2013)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Pegasus: Designing a Distributed Key Value System (Arch summit beijing-2016)
Hive+Tez: A performance deep dive
Hive+Tez: A performance deep dive
[234]멀티테넌트 하둡 클러스터 운영 경험기
[234]멀티테넌트 하둡 클러스터 운영 경험기
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Introduction to Redis
Introduction to Redis
HBase Application Performance Improvement
HBase Application Performance Improvement
Need for Time series Database
Need for Time series Database
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
Don’t Forget About Your Past—Optimizing Apache Druid Performance With Neil Bu...
Hive User Meeting August 2009 Facebook
Hive User Meeting August 2009 Facebook
ORC File - Optimizing Your Big Data
ORC File - Optimizing Your Big Data
Optimizing Apache Spark SQL Joins
Optimizing Apache Spark SQL Joins
Care and Feeding of Catalyst Optimizer
Care and Feeding of Catalyst Optimizer
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
[212]C3, 데이터 처리에서 서빙까지 가능한 하둡 클러스터
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
hive HBase Metastore - Improving Hive with a Big Data Metadata Storage
MongoDB Fundamentals
MongoDB Fundamentals
HBase Advanced - Lars George
HBase Advanced - Lars George
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
Redis + Apache Spark = Swiss Army Knife Meets Kitchen Sink
En vedette
Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010
Tim Y
新浪微博分布式缓存与队列-2013版
新浪微博分布式缓存与队列-2013版
XiaoJun Hong
Feed服务架构-新浪微博新员工培训议题
Feed服务架构-新浪微博新员工培训议题
XiaoJun Hong
高性能队列Fqueue的设计和使用实践
高性能队列Fqueue的设计和使用实践
孙立
人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析
isnull
设计模式分享
设计模式分享
michaelsh
Java常见问题排查
Java常见问题排查
bluedavy lin
Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言
Feng Yu
13 tips for a highly engaging feed
13 tips for a highly engaging feed
Thierry Schellenbach
Depoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PF
Luis Nassif
20140717 about openstreetmap
20140717 about openstreetmap
Taichi Furuhashi
Lesson 7 1 Part 2 1
Lesson 7 1 Part 2 1
mradcliffe
Fleury no inferno
Fleury no inferno
Luis Nassif
About OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese Geographers
Taichi Furuhashi
Miguli Basic
Miguli Basic
Komunalne usluge Djurdjevac d.o.o.
Space hackathon5
Space hackathon5
Taichi Furuhashi
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Farid Mokhtar Noriega
Alfredorodriguez
Alfredorodriguez
Piebel
Michigan Ppt[1]
Michigan Ppt[1]
mduncan
Hi France24 Iht Harris Poll Leaders Barometer April2009
Hi France24 Iht Harris Poll Leaders Barometer April2009
Luis Nassif
En vedette
(20)
Build scalable microblog qcon beijing 2010
Build scalable microblog qcon beijing 2010
新浪微博分布式缓存与队列-2013版
新浪微博分布式缓存与队列-2013版
Feed服务架构-新浪微博新员工培训议题
Feed服务架构-新浪微博新员工培训议题
高性能队列Fqueue的设计和使用实践
高性能队列Fqueue的设计和使用实践
人人网技术经理张铁安 Feed系统结构浅析
人人网技术经理张铁安 Feed系统结构浅析
设计模式分享
设计模式分享
Java常见问题排查
Java常见问题排查
Erlang分布式系统的的领域语言
Erlang分布式系统的的领域语言
13 tips for a highly engaging feed
13 tips for a highly engaging feed
Depoimento de Amaury Jr ne PF
Depoimento de Amaury Jr ne PF
20140717 about openstreetmap
20140717 about openstreetmap
Lesson 7 1 Part 2 1
Lesson 7 1 Part 2 1
Fleury no inferno
Fleury no inferno
About OSM japan at the Association of Japanese Geographers
About OSM japan at the Association of Japanese Geographers
Miguli Basic
Miguli Basic
Space hackathon5
Space hackathon5
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Collaborative 3D Modelling and Printing: What you See is not Directly What Yo...
Alfredorodriguez
Alfredorodriguez
Michigan Ppt[1]
Michigan Ppt[1]
Hi France24 Iht Harris Poll Leaders Barometer April2009
Hi France24 Iht Harris Poll Leaders Barometer April2009
Similaire à 微博cache设计谈
Timyang新浪微博设计谈
Timyang新浪微博设计谈
Cevin Cheung
Tup2 新浪杨卫华
Tup2 新浪杨卫华
yongboy
杨卫华:微博cache设计浅谈
杨卫华:微博cache设计浅谈
Leechael
Kiosk / PHP
Kiosk / PHP
Basuke Suzuki
Cache on Delivery
Cache on Delivery
SensePost
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
Korea Sdec
BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011
Yiwei Ma
04 standard class library c#
04 standard class library c#
Victor Matyushevskyy
ログ収集プラットフォーム開発におけるElasticsearchの運用
ログ収集プラットフォーム開発におけるElasticsearchの運用
LINE Corporation
Amazon Aurora로 안전하게 migration 하기
Amazon Aurora로 안전하게 migration 하기
Jesang Yoon
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
DATAVERSITY
Introducing Amplify
Introducing Amplify
appendTo
Html5: Something wicked this way comes (Hack in Paris)
Html5: Something wicked this way comes (Hack in Paris)
Krzysztof Kotowicz
Nuxeo World Session: CMIS - What's Next?
Nuxeo World Session: CMIS - What's Next?
Nuxeo
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy Hawkins
NETWAYS
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
NETWAYS
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
NETWAYS
ActiveRecordのテーブル名の話
ActiveRecordのテーブル名の話
Akira Ohta
初心者向けGAE/Java説明資料
初心者向けGAE/Java説明資料
Shinichi Ogawa
iPhone dev intro
iPhone dev intro
Vonbo
Similaire à 微博cache设计谈
(20)
Timyang新浪微博设计谈
Timyang新浪微博设计谈
Tup2 新浪杨卫华
Tup2 新浪杨卫华
杨卫华:微博cache设计浅谈
杨卫华:微博cache设计浅谈
Kiosk / PHP
Kiosk / PHP
Cache on Delivery
Cache on Delivery
SDEC2011 NoSQL concepts and models
SDEC2011 NoSQL concepts and models
BuilHigh Performance Weibo Platform-Qcon2011
BuilHigh Performance Weibo Platform-Qcon2011
04 standard class library c#
04 standard class library c#
ログ収集プラットフォーム開発におけるElasticsearchの運用
ログ収集プラットフォーム開発におけるElasticsearchの運用
Amazon Aurora로 안전하게 migration 하기
Amazon Aurora로 안전하게 migration 하기
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
Introducing Amplify
Introducing Amplify
Html5: Something wicked this way comes (Hack in Paris)
Html5: Something wicked this way comes (Hack in Paris)
Nuxeo World Session: CMIS - What's Next?
Nuxeo World Session: CMIS - What's Next?
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSDC 2013 | Introduction into Chef by Andy Hawkins
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica Sarbu
ActiveRecordのテーブル名の話
ActiveRecordのテーブル名の話
初心者向けGAE/Java説明資料
初心者向けGAE/Java説明資料
iPhone dev intro
iPhone dev intro
Dernier
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
RTylerCroy
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
UK Journal
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Neo4j
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Puma Security, LLC
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Michael W. Hawkins
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
Principled Technologies
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Results
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
wesley chun
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
debabhi2
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Dernier
(20)
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
微博cache设计谈
1.
cache
@TimYang
2.
Agenda • Feed •
Cache
3.
• •
, status feed •
4.
Feed •
feed • Push( ) • Pull( ) •
5.
Push •
feed • Inbox: • Outbox: • inbox( ) • Inbox( )
6.
Push • • •
7.
Pull •
outbox( ) • Inbox( ) • Online computation
8.
Pull • •
9.
• • •
10.
Cache memory is the
new disk, and disk is the new tape. for "real-time" web applications, and systems that require massive scalability - Jim Gray
11.
Cache
12.
cache
Weibo cache arch Inbox hot cache Outbox Vector cache Archive cache Social Following Followers users Graph Content Hot cache Total
13.
Inbox •
“ ” • cache id list • (hot cache) • get set • redis list/set
14.
Outbox • •
id list(e.g. 100) • id LRU
15.
Social Graph cache •
Following ids • Followers • • • delete all
16.
Content cache •
cache • • • • Open API xml, json...
17.
Cache • •
18.
Update Workflow
Update status Content cache Hot Inbox Vector Outbox vector Content cache replication
19.
•
outbox hot vector • followers list • inbox
20.
feed
Home timeline Workflow home_timeline aggregator Content hot Inbox cache Outbox Vector cache Inbox archive Inbox archive Content cache
21.
feed •
inbox cache • • , following • id list feed
22.
cache • • hot keys • •
mutex
23.
•
Content cache • multi get n feed(n = items/ , e.g. 50) • cache = n * (feed + , e.g. 2k) • 1,000 /
24.
• 1,000
800Mbps •1 8Gbps
25.
•
1G 300~400Mbps • • local cache • •
26.
hot keys • content
cache of • create local cache 1. get user_yaochen_local 2. get user_yaochen 1. set user_yaochen_local:value 3. delete all
27.
cache •
key memcache • • LRU cache, • Memory storage
28.
mutex •
content item cache • db • db • key_mutex key • mutex
29.
•
• S2 • S2 •
30.
S2 • Web 2.0
Social • What’s S2 • Social Platform & • Social Applications • www.s2forum.org
31.
• • Sina App
Engine • • •
32.
Q&A • email: iso1600@gmail.com •
@TimYang
Notes de l'éditeur
Télécharger maintenant