Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
2009.4

hongqn@douban.com
•   2005 3

•

•

•
• 2.9M            1/4
•
• 23M        /        500~600/sec
• 23      PC      (1U*15/2U*8)
  • 12
  • 38G memcached
•        1U         (frodo)
    •        AMD Athlon 64 1.8GHz
    •   1G       160G SATA*2
•   Gentoo Linux
•   MySQL 5
• ...
Gentoo Linux
•
    • emerge mysql
    • ebuild          patch
•
•
    • GLSA(Gentoo Linux Security Advisories)
MySQL

•   The world’s most popular open source database

•            /             ==> MyISAM

•                ==> Inno...
Python

•
•   Battery Included

•
•
    •   CPUG: http://python.cn/
Quixote
•                           REST         URL
•              Django, TurboGears, Pylons
              ZOPE
• http:/...
Lighttpd
•
•       SCGI
    • SCGI:               FastCGI
      Quixote
•                  80        lighttpd
            ...
Memcache
•                                           MySQL
•     libmemcache   python                          Pyrex
     ...
Internet




                  Lighttpd

                SCGI



          App

                       FS




MySQL   Memc...
• 1.2M   /
• IO
•
•                1U
    •   pippin       meriadoc (      merry)
    •         , 4G          250G SATA*3
•
•               ...
Internet




        Lighttpd (#$)                               DNS
                                        HTTP
        ...
•
    • innodb_buffer_pool_size
•
•              IP
• 1.5M   /
•
• IP
•                            IP(BGP)
•                         (arwen)
    • 74G 1w   SATA * 3
    •
•
Internet




                                                               Data
               Lighttpd                  ...
• 2M              /
•                     IO
  •
       , etc...
•
•                               4G   250G SATA*3
•                                10000
    •   mod_rewrite       URL
•   ...
Internet

                                                                store.farm
                                     ...
•   store          farmr
•                replicate delay
    •
    •

    •                          cache
        •
•

 ...
replicate delay
def get_subject(sid):
   sbj = mc.get(‘s:’+sid)
   if sbj is None:
      sbj = flush_subject(sid, store.fa...
• 2.5M       /
•
  •      /
• SATA
•
•   Scale Up               1U
    •   16G        147G SCSI *2 + 500G SATA
    •   SCSI     RAID-0
•       MySQL Slave
•   ...
Internet
                                                     Sphinx


                                                   ...
• 5.2M   /
•
• Web        IO


•
•
•                             :)
    •
    •
    •
•        3 1U             4        32G        1T SATA * 3
•            ...
Internet


                          www.douban.com          otho.douban.com


                                           ...
write




                                      !quot;#$%
                replicate
                                      ...
• 6.4M       /   (5M PV)
•
  •
 • CPU   memcache          /
•
    •   lighttpd     mod_scgi     round-robin
    •   lighttpd 1.5
    •   mod_proxy
        •   proxy.balance = fair (l...
Internet


                                                     Lighttpd

                                                ...
• 11M      /      3
•
• Sphinx
•
•
           load
•
    •
    •
• Sphinx -> Xapian
• MogileFS
!quot;
             Master

                       replicate

replicate
                                                !q...
•
•   store.farm[r] -> store.get_cursor(table=‘xxx’,
    ro=True/False)
    def flush_subject(sid, ro=False):
        curs...
Internet

                         upload.douban.com otho.douban.com




                                                 ...
•   libmemcache -> libmemcached              consistent
    hash    memcache
    •       libmemcached consistent hash     ...
Consistent Hash




                                charlee http://
                  tech.idv2.com/2008/07/24/
          ...
Internet


                                                                  App
                                         ...
• 13M   /
•                 MogileFS
  •         Tracker DB


•
•       8
    • 32G            CPU
    • (300G SCSI×2 + 1T SATA) × 3
    • (1T SATA × 3) × 5
    •6          2
•       Dou...
DoubanFS
•                            hash


•      hash                              hash


•                          Me...
Merkle Tree
Internet

                         upload.douban.com otho.douban.com




                                                 ...
• 16M        /
•
• DoubanFS       IO
•
•       DoubanDB
    •
    •                 MySQL


• MySQL Master
  • failover
  • replicate delay
DoubanDB
•         Key-Value
•     Amazon Dynamo
•              set(key, value), get(key), delete(key)
•   memcache
•     ...
charlee http://
tech.idv2.com/2008/07/24/
memcached-004/
!quot;
             Master1

                        replicate

replicate
                                                ...
•       DoubanFS
    •
    •
• ngnix
• www.douban.com       LVS
• RabbitMQ         spread
•
•              profile
•   memcache            cache


•                join
•
•
•
豆瓣技术架构的发展历程 @ QCon Beijing 2009
Prochain SlideShare
Chargement dans…5
×

豆瓣技术架构的发展历程 @ QCon Beijing 2009

39 031 vues

Publié le

Publié dans : Technologie
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • 豆瓣技术架构的发展历程
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • sphinx -> xapian (sphinx定制性不够)
    lighttpd -> nginx 做 load banlance
    spread -> nginx 记录日志
    RabbitMQ代替spread
    libmemcache -> libmemcached
    使用 consistent hash降低memcache调整代价
    scgi而非fast cgi
    MogileFS
    开发DoubanFS: 与MogileFS,读性能3倍,写性能50倍 (NoSql的思想?): Merkle Tree
    开发DoubanDB
    LVS
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

豆瓣技术架构的发展历程 @ QCon Beijing 2009

  1. 1. 2009.4 hongqn@douban.com
  2. 2. • 2005 3 • • •
  3. 3. • 2.9M 1/4 • • 23M / 500~600/sec • 23 PC (1U*15/2U*8) • 12 • 38G memcached
  4. 4. • 1U (frodo) • AMD Athlon 64 1.8GHz • 1G 160G SATA*2 • Gentoo Linux • MySQL 5 • Quixote (a Python web framework) • Lighttpd + SCGI (shire) • Memcached (!)
  5. 5. Gentoo Linux • • emerge mysql • ebuild patch • • • GLSA(Gentoo Linux Security Advisories)
  6. 6. MySQL • The world’s most popular open source database • / ==> MyISAM • ==> InnoDB • Replicate for backup
  7. 7. Python • • Battery Included • • • CPUG: http://python.cn/
  8. 8. Quixote • REST URL • Django, TurboGears, Pylons ZOPE • http://www.douban.com/subject/1000001 # luz/subject/__init__.py def _q_lookup(request, name): subject = get_subject(name) return lambda req: subject_ui(req, subject) # luz/subject/subject_ui.ptl def subject_ui [html] (request, subject): site_header(request) “<h1>%s</h1>” % subject.title site_footer(request)
  9. 9. Lighttpd • • SCGI • SCGI: FastCGI Quixote • 80 lighttpd SCGI localhost Quixote
  10. 10. Memcache • MySQL • libmemcache python Pyrex python 3x+ def get_subject(subject_id): subject = mc.get(‘s:’+subject_id) if subject is None: store.farm.execute(“select xxx, xxx from subject where id=%s”, subject_id) subject = Subject(*store.farm.fetchone()) mc.set(‘s:’+subject_id, subject) return subject
  11. 11. Internet Lighttpd SCGI App FS MySQL Memcache Static Files
  12. 12. • 1.2M / • IO •
  13. 13. • 1U • pippin meriadoc ( merry) • , 4G 250G SATA*3 • • IP DNS IP -_-b • frodo (subversion, trac, etc...)
  14. 14. Internet Lighttpd (#$) DNS HTTP Proxy SCGI App Lighttpd (!quot;) FS Static Files Memcache MySQL
  15. 15. • • innodb_buffer_pool_size • • IP
  16. 16. • 1.5M / • • IP
  17. 17. • IP(BGP) • (arwen) • 74G 1w SATA * 3 • •
  18. 18. Internet Data Lighttpd Mining read SCGI write App Replicate MySQL MySQL Master Slave Memcache Static Files
  19. 19. • 2M / • IO • , etc... •
  20. 20. • 4G 250G SATA*3 • 10000 • mod_rewrite URL • lighttpd mod_memcache • IO • web • •
  21. 21. Internet store.farm App MySQL SCGI Master Lighttpd Memcache Replicate store.farmr HTTP Proxy WebDAV Web Service Replicate Lighttpd (w/ mod_memcache) MySQL write Slave Spiders Lighttpd WebDAV Memcache Static Files !quot;#$% read Data Mining MySQL Slave
  22. 22. • store farmr • replicate delay • • • cache • • • ...... but it works
  23. 23. replicate delay def get_subject(sid): sbj = mc.get(‘s:’+sid) if sbj is None: sbj = flush_subject(sid, store.farmr) return sbj def flush_subject(sid, cursor=None): cursor = cursor or store.farm cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject def update_subject(subject, props): store.farm.execute(“update subject ...”) store.farm.connection.commit() flush_subject(subject.id, store.farm)
  24. 24. • 2.5M / • • / • SATA •
  25. 25. • Scale Up 1U • 16G 147G SCSI *2 + 500G SATA • SCSI RAID-0 • MySQL Slave • memcached • MyISAM InnoDB • • Sphinx
  26. 26. Internet Sphinx MySQL Master store.farm Web Service Replicate Lighttpd SCGI App HTTP Proxy Memcache Lighttpd (w/ mod_memcache) WebDAV store.farmr Memcache Lighttpd Web Service WebDAV MySQL Static Files Slave Spiders Memcache Memcache
  27. 27. • 5.2M / • • Web IO • •
  28. 28. • :) • • • • 3 1U 4 32G 1T SATA * 3 • otho.douban.com lotho.douban.com • lighttpd 1.5 with aio support • LVS • Scale Up: 4G -> 8G
  29. 29. Internet www.douban.com otho.douban.com LVS LB (Master) Lighttpd Lighttpd 1.5 (w/ mod_cache) HTTP Proxy Lighttpd Keepalived LVS LB (backup) Lighttpd Lighttpd 1.5 WebDAV (w/ mod_cache) Static Files
  30. 30. write !quot;#$% replicate read Data Mining MySQL MySQL Slave Master read Replicate write !quot;#$% Data Mining MySQL Slave
  31. 31. • 6.4M / (5M PV) • • • CPU memcache /
  32. 32. • • lighttpd mod_scgi round-robin • lighttpd 1.5 • mod_proxy • proxy.balance = fair (load based, passive balancing) • • spread
  33. 33. Internet Lighttpd SCGI HTTP Proxy App Memcache Lighttpd HTTP Proxy spread HTTP Proxy Lighttpd Lighttpd Log Aggregator SCGI spread Lighttpd Memcache App WebDAV Static Files
  34. 34. • 11M / 3 • • Sphinx • • load
  35. 35. • • • • Sphinx -> Xapian • MogileFS
  36. 36. !quot; Master replicate replicate !quot; %&'() Slave #$ Master read replicate Data #$ read Slave Mining write *+,- !quot; Master Slave replicate *+,- Slave write replicate Data %&'() Mining #$ Slave
  37. 37. • • store.farm[r] -> store.get_cursor(table=‘xxx’, ro=True/False) def flush_subject(sid, ro=False): cursor = store.get_cursor(table=‘subject’, ro=ro) cursor.execute(“select ... from subject”) subject = Subject(*cursor.fetchone()) mc.set(‘s:’+sid, subject) return subject •
  38. 38. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage MogileFS Tracker Gateway App MogileFS MogileFS Node Node MogileFS Master
  39. 39. • libmemcache -> libmemcached consistent hash memcache • libmemcached consistent hash bug • CPU • libmemcached failover bug • nginx lighttpd load balance • spread • nginx
  40. 40. Consistent Hash charlee http:// tech.idv2.com/2008/07/24/ memcached-004/
  41. 41. Internet App SCGI Lighttpd HTTP Proxy Lighttpd HTTP Proxy App HTTP Proxy SCGI Lighttpd HTTP Proxy Nginx Lighttpd HTTP Proxy App Lighttpd SCGI Lighttpd WebDAV Static Files
  42. 42. • 13M / • MogileFS • Tracker DB •
  43. 43. • 8 • 32G CPU • (300G SCSI×2 + 1T SATA) × 3 • (1T SATA × 3) × 5 •6 2 • DoubanFS
  44. 44. DoubanFS • hash • hash hash • Merkle Tree • consistent hash • WebDAV • MogileFS 3 50
  45. 45. Merkle Tree
  46. 46. Internet upload.douban.com otho.douban.com nginx signed POST form Uploader (w/ proxy_store) HTTP redirect FileStorage Gateway App DoubanFS DoubanFS Node Node
  47. 47. • 16M / • • DoubanFS IO •
  48. 48. • DoubanDB • • MySQL • MySQL Master • failover • replicate delay
  49. 49. DoubanDB • Key-Value • Amazon Dynamo • set(key, value), get(key), delete(key) • memcache • Merkle Tree • Consistent Hash • TokyoCabinet • DoubanDB • DoubanFS 2.0 DoubanDB
  50. 50. charlee http:// tech.idv2.com/2008/07/24/ memcached-004/
  51. 51. !quot; Master1 replicate replicate !quot; %&'() Slave #$ Master1 read replicate Data #$ read Slave Mining write *+,- !quot; Master Master2 replicate *+,- Slave write replicate Data %&'() Mining #$ Master2
  52. 52. • DoubanFS • • • ngnix • www.douban.com LVS • RabbitMQ spread
  53. 53. • • profile • memcache cache • join • • •

×