SlideShare une entreprise Scribd logo
1  sur  571
Django in the Real World

Jacob Kaplan-Moss

OSCON 2009
http://jacobian.org/TN
Jacob Kaplan-Moss
http://jacobian.org / jacob@jacobian.org / @jacobian

              Lead Developer, Django

            Partner, Revolution Systems




                                                       2
Shameless plug:



    http://revsys.com/




                         3
Hat tip:
James Bennett (http://b-list.org)




                                    4
So you’ve written a
  Django site…


                      5
… now what?



              6
http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                      7
•   API Metering




                   http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                         7
•   API Metering
•   Backups & Snapshots




                   http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                         7
•   API Metering
•   Backups & Snapshots
•   Counters




                   http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                         7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools




                   http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                         7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments




                    http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                          7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)




                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                           7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing




                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                           7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks



                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                           7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management


                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                           7
•   API Metering
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                           7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades                                     •   Ranging
     •   Rolling deployments
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades                                     •   Ranging
     •   Rolling deployments                                     •   Geo
     •   Multiple versions (selective beta)
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades                                     •   Ranging
     •   Rolling deployments                                     •   Geo
     •   Multiple versions (selective beta)                 •   Sharding
     •   Bucket Testing
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades                                     •   Ranging
     •   Rolling deployments                                     •   Geo
     •   Multiple versions (selective beta)                 •   Sharding
     •   Bucket Testing                                     •   Smart Caching
     •   Rollbacks
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
•   API Metering                                            •   Distributed Log storage, analysis
•   Backups & Snapshots                                     •   Graphing
•   Counters                                                •   HTTP Caching
•   Cloud/Cluster Management Tools                          •   Input/Output Filtering
     •   Instrumentation/Monitoring                         •   Memory Caching
     •   Failover                                           •   Non-relational Key Stores
     •   Node addition/removal and hashing                  •   Rate Limiting
     •   Auto-scaling for cloud resources                   •   Relational Storage
•   CSRF/XSS Protection                                     •   Queues
•   Data Retention/Archival                                 •   Rate Limiting
•   Deployment Tools                                        •   Real-time messaging (XMPP)
     •   Multiple Devs, Staging, Prod                       •   Search
     •   Data model upgrades                                     •   Ranging
     •   Rolling deployments                                     •   Geo
     •   Multiple versions (selective beta)                 •   Sharding
     •   Bucket Testing                                     •   Smart Caching
     •   Rollbacks                                               •   Dirty-table management
     •   CDN Management
•   Distributed File Storage
                     http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
                                                                                                7
The bare minimum:




                    8
The bare minimum:
• Test.




                    8
The bare minimum:
• Test.
• Structure for deployment.




                              8
The bare minimum:
• Test.
• Structure for deployment.
• Use deployment tools.




                              8
The bare minimum:
• Test.
• Structure for deployment.
• Use deployment tools.
• Design a production environment.




                                     8
The bare minimum:
• Test.
• Structure for deployment.
• Use deployment tools.
• Design a production environment.
• Monitor.



                                     8
The bare minimum:
• Test.
• Structure for deployment.
• Use deployment tools.
• Design a production environment.
• Monitor.
• Tune.

                                     8
Testing



          9
“   Tests are the
Programmer’s stone,
transmuting fear into



                      ”
     boredom.

              — Kent Beck

                        10
Hardcore TDD



               11
“
    I don’t do test driven
development. I do stupidity
driven testing… I wait until
I do something stupid, and




                           ”
  then write tests to avoid
        doing it again.

                 — Titus Brown

                               12
Whatever happens, don’t let
your test suite break thinking,
“I’ll go back and fix this later.”



                                13
Unit testing




Functional/behavior
testing




Browser testing
                      14
Unit testing          unittest




Functional/behavior
testing




Browser testing
                                 14
Unit testing          unittest



                      doctest
Functional/behavior
testing




Browser testing
                                 14
Unit testing                  unittest



                              doctest
Functional/behavior
testing
                      django.test.Client, Twill



Browser testing
                                                  14
Unit testing                  unittest



                              doctest
Functional/behavior
testing
                      django.test.Client, Twill



Browser testing         Windmill, Selenium
                                                  14
You need them all.



                     15
Testing Django




                 16
Testing Django

• Unit tests (unittest)




                          16
Testing Django

• Unit tests (unittest)
• Doctests (doctest)




                          16
Testing Django

• Unit tests (unittest)
• Doctests (doctest)
• Fixtures




                          16
Testing Django

• Unit tests (unittest)
• Doctests (doctest)
• Fixtures
• Test client




                          16
Testing Django

• Unit tests (unittest)
• Doctests (doctest)
• Fixtures
• Test client
• Email capture


                          16
Unit tests




             17
Unit tests
• “Whitebox” testing




                       17
Unit tests
• “Whitebox” testing
• Verify the small functional units of your
  app




                                              17
Unit tests
• “Whitebox” testing
• Verify the small functional units of your
  app
• Very fine-grained




                                              17
Unit tests
• “Whitebox” testing
• Verify the small functional units of your
  app
• Very fine-grained
• Familier to most programmers (JUnit,
  NUnit, etc.)



                                              17
Unit tests
• “Whitebox” testing
• Verify the small functional units of your
  app
• Very fine-grained
• Familier to most programmers (JUnit,
  NUnit, etc.)
• Provided in Python by unittest

                                              17
django.test.TestCase




                       18
django.test.TestCase

• Fixtures.




                       18
django.test.TestCase

• Fixtures.
• Test client.




                       18
django.test.TestCase

• Fixtures.
• Test client.
• Email capture.




                       18
django.test.TestCase

• Fixtures.
• Test client.
• Email capture.
• Database management.




                         18
django.test.TestCase

• Fixtures.
• Test client.
• Email capture.
• Database management.
• Slower than unittest.TestCase.


                                   18
class StoryAddViewTests(TestCase):
   fixtures = ['authtestdata', 'newsbudget_test_data']
   urls = 'newsbudget.urls'

  def test_story_add_get(self):
    r = self.client.get('/budget/stories/add/')
    self.assertEqual(r.status_code, 200)
      …

  def test_story_add_post(self):
    data = {
       'title': 'Hungry cat is hungry',
       'date': '2009-01-01',
    }
    r = self.client.post('/budget/stories/add/', data)
    self.assertEqual(r.status_code, 302)
    …



                                                         19
Doctests




           20
Doctests
• Easy to write & read.




                          20
Doctests
• Easy to write & read.
• Produces self-documenting code.




                                    20
Doctests
• Easy to write & read.
• Produces self-documenting code.
• Great for cases that only use assertEquals.




                                                20
Doctests
• Easy to write & read.
• Produces self-documenting code.
• Great for cases that only use assertEquals.
• Somewhere between unit tests and
  functional tests.




                                                20
Doctests
• Easy to write & read.
• Produces self-documenting code.
• Great for cases that only use assertEquals.
• Somewhere between unit tests and
  functional tests.
• Difficult to debug.



                                                20
Doctests
• Easy to write & read.
• Produces self-documenting code.
• Great for cases that only use assertEquals.
• Somewhere between unit tests and
  functional tests.
• Difficult to debug.
• Don’t always provide useful test failures.

                                                20
class Choices(object):
   """
   Easy declarative "choices" tool::

        >>> STATUSES = Choices("Live", "Draft")

        # Acts like a choices list:
        >>> list(STATUSES)
        [(1, 'Live'), (2, 'Draft')]

        # Easily convert from code to verbose:
        >>> STATUSES.verbose(1)
        'Live'

        # ... and vice versa:
        >>> STATUSES.code("Draft")
        2

  """
  …



                                                  21
****************************************************
File "utils.py", line 150, in __main__.Choices
Failed example:
   STATUSES.verbose(1)
Expected:
   'Live'
Got:
   'Draft'
****************************************************




                                                       22
Functional tests




                   23
Functional tests
• a.k.a “Behavior Driven Development.”




                                         23
Functional tests
• a.k.a “Behavior Driven Development.”
• “Blackbox,” holistic testing.




                                         23
Functional tests
• a.k.a “Behavior Driven Development.”
• “Blackbox,” holistic testing.
• All the hardcore TDD folks look down on
  functional tests.




                                            23
Functional tests
• a.k.a “Behavior Driven Development.”
• “Blackbox,” holistic testing.
• All the hardcore TDD folks look down on
  functional tests.
• But they keep your boss happy.




                                            23
Functional tests
• a.k.a “Behavior Driven Development.”
• “Blackbox,” holistic testing.
• All the hardcore TDD folks look down on
  functional tests.
• But they keep your boss happy.
• Easy to find problems; harder to find the
  actual bug.

                                            23
Functional testing
tools




                     24
Functional testing
tools
• django.test.Client




                       24
Functional testing
tools
• django.test.Client
• webunit




                       24
Functional testing
tools
• django.test.Client
• webunit
• Twill




                       24
Functional testing
tools
• django.test.Client
• webunit
• Twill
• ...



                       24
django.test.Client




                     25
django.test.Client

• Test the whole request path without
  running a web server.




                                        25
django.test.Client

• Test the whole request path without
  running a web server.
• Responses provide extra information
  about templates and their contexts.




                                        25
class StoryAddViewTests(TestCase):
   fixtures = ['authtestdata', 'newsbudget_test_data']
   urls = 'newsbudget.urls'

  def test_story_add_get(self):
    r = self.client.get('/budget/stories/add/')
    self.assertEqual(r.status_code, 200)
  …

  def test_story_add_post(self):
    data = {
       'title': 'Hungry cat is hungry',
       'date': '2009-01-01',
    }
    r = self.client.post('/budget/stories/add/', data)
    self.assertEqual(r.status_code, 302)
    …




                                                         26
Web browser testing




                      27
Web browser testing

• The ultimate in functional testing for
  web applications.




                                           27
Web browser testing

• The ultimate in functional testing for
  web applications.
• Run test in a web browser.




                                           27
Web browser testing

• The ultimate in functional testing for
  web applications.
• Run test in a web browser.
• Can verify JavaScript, AJAX; even CSS.




                                           27
Web browser testing

• The ultimate in functional testing for
  web applications.
• Run test in a web browser.
• Can verify JavaScript, AJAX; even CSS.
• Test your site across supported browsers.



                                              27
Browser testing tools




                        28
Browser testing tools


• Selenium




                        28
Browser testing tools


• Selenium
• Windmill




                        28
“Exotic” testing




                   29
“Exotic” testing

• Static source analysis.




                            29
“Exotic” testing

• Static source analysis.
• Smoke testing (crawlers and spiders).




                                          29
“Exotic” testing

• Static source analysis.
• Smoke testing (crawlers and spiders).
• Monkey testing.




                                          29
“Exotic” testing

• Static source analysis.
• Smoke testing (crawlers and spiders).
• Monkey testing.
• Load testing.




                                          29
“Exotic” testing

• Static source analysis.
• Smoke testing (crawlers and spiders).
• Monkey testing.
• Load testing.
• ...


                                          29
30
Further resources




                    31
Further resources

• Windmill talk here at OSCON
 http://bit.ly/14tkrd




                                31
Further resources

• Windmill talk here at OSCON
 http://bit.ly/14tkrd
• Django testing documentation
 http://bit.ly/django-testing




                                 31
Further resources

• Windmill talk here at OSCON
 http://bit.ly/14tkrd
• Django testing documentation
 http://bit.ly/django-testing
• Python Testing Tools Taxonomy
 http://bit.ly/py-testing-tools



                                  31
Structuring
applications for reuse


                         32
Designing for reuse




                      33
Designing for reuse

• Do one thing, and do it well.




                                  33
Designing for reuse

• Do one thing, and do it well.
• Don’t be afraid of multiple apps.




                                      33
Designing for reuse

• Do one thing, and do it well.
• Don’t be afraid of multiple apps.
• Write for flexibility.




                                      33
Designing for reuse

• Do one thing, and do it well.
• Don’t be afraid of multiple apps.
• Write for flexibility.
• Build to distribute.




                                      33
Designing for reuse

• Do one thing, and do it well.
• Don’t be afraid of multiple apps.
• Write for flexibility.
• Build to distribute.
• Extend carefully.


                                      33
1.
Do one thing, and do it well.




                                34
Application == encapsulation




                           35
Focus




        36
Focus

• Ask yourself: “What does this
  application do?”




                                  36
Focus

• Ask yourself: “What does this
  application do?”
• Answer should be one or two
  short sentences.




                                  36
Good focus




             37
Good focus

• “Handle storage of users and
  authentication of their identities.”




                                         37
Good focus

• “Handle storage of users and
  authentication of their identities.”
• “Allow content to be tagged, del.icio.us
  style, with querying by tags.”




                                             37
Good focus

• “Handle storage of users and
  authentication of their identities.”
• “Allow content to be tagged, del.icio.us
  style, with querying by tags.”
• “Handle entries in a weblog.”



                                             37
Bad focus




            38
Bad focus


• “Handle entries in a weblog, and users
  who post them, and their authentication,
  and tagging and categorization, and some
  flat pages for static content, and...”




                                             38
Warning signs




                39
Warning signs

• Lots of files.




                  39
Warning signs

• Lots of files.
• Lots of modules.




                     39
Warning signs

• Lots of files.
• Lots of modules.
• Lots of models.




                     39
Warning signs

• Lots of files.
• Lots of modules.
• Lots of models.
• Lots of code.



                     39
Small is good




                40
Small is good

• Many great Django apps are very small.




                                           40
Small is good

• Many great Django apps are very small.
• Even a lot of “simple” Django sites
  commonly have a dozen or more
  applications in INSTALLED_APPS.




                                           40
Small is good

• Many great Django apps are very small.
• Even a lot of “simple” Django sites
  commonly have a dozen or more
  applications in INSTALLED_APPS.
• If you’ve got a complex site and a short
  application list, something’s probably wrong.



                                                  40
Approach features skeptically




                                41
Approach features skeptically


• What does the application do?




                                  41
Approach features skeptically


• What does the application do?
• Does this feature have anything to do
  with that?




                                          41
Approach features skeptically


• What does the application do?
• Does this feature have anything to do
  with that?
• No? Don’t add it.



                                          41
2.
Don’t be afraid of many apps.




                                42
The monolith anti-pattern




                            43
The monolith anti-pattern


• The “application” is the whole site.




                                         43
The monolith anti-pattern


• The “application” is the whole site.
• Re-use? YAGNI.




                                         43
The monolith anti-pattern


• The “application” is the whole site.
• Re-use? YAGNI.
• Plugins that hook into the “main” application.




                                                   43
The monolith anti-pattern


• The “application” is the whole site.
• Re-use? YAGNI.
• Plugins that hook into the “main” application.
• Heavy use of middleware-like concepts.




                                                   43
(I blame Rails)




                  44
The Django mindset




                     45
The Django mindset

• Application: some bit of functionality.




                                            45
The Django mindset

• Application: some bit of functionality.
• Site: several applications.




                                            45
The Django mindset

• Application: some bit of functionality.
• Site: several applications.
• Spin off new “apps” liberally.




                                            45
The Django mindset

• Application: some bit of functionality.
• Site: several applications.
• Spin off new “apps” liberally.
• Develop a suite of apps ready for when
  they’re needed.



                                            45
Django encourages this




                     46
Django encourages this

• INSTALLED_APPS




                     46
Django encourages this

• INSTALLED_APPS
• Applications are just Python packages,
  not some Django-specific “app” or
  “plugin.”




                                           46
Django encourages this

• INSTALLED_APPS
• Applications are just Python packages,
  not some Django-specific “app” or
  “plugin.”
• Abstractions like django.contrib.sites
  make you think about this as you develop.


                                              46
Spin off a new app?




                      47
Spin off a new app?


• Is this feature unrelated to the app’s focus?




                                                  47
Spin off a new app?


• Is this feature unrelated to the app’s focus?
• Is it orthogonal to the rest of the app?




                                                  47
Spin off a new app?


• Is this feature unrelated to the app’s focus?
• Is it orthogonal to the rest of the app?
• Will I need similar functionality again?




                                                  47
The ideal:



             48
I need a contact form



                        49
urlpatterns = ('',
   …
   (r'^contact/', include('contact_form.urls')),
   …
)




                                                   50
Done.
(http://bitbucket.org/ubernostrum/django-contact-form/)




                                                          51
But… what about…




                   52
But… what about…

• Site A wants a contact form that just
  collects a message.




                                          52
But… what about…

• Site A wants a contact form that just
  collects a message.
• Site B’s marketing department wants a
  bunch of info.




                                          52
But… what about…

• Site A wants a contact form that just
  collects a message.
• Site B’s marketing department wants a
  bunch of info.
• Site C wants to use Akismet to filter
  automated spam.


                                          52
53
53
3.
Write for flexibility.




                        54
Common sense




               55
Common sense


• Sane defaults.




                   55
Common sense


• Sane defaults.
• Easy overrides.




                    55
Common sense


• Sane defaults.
• Easy overrides.
• Don’t set anything in stone.




                                 55
Forms




        56
Forms


• Supply a form class.




                         56
Forms


• Supply a form class.
• Let users specify their own.




                                 56
Templates




            57
Templates


• Specify a default template.




                                57
Templates


• Specify a default template.
• Let users specify their own.




                                 57
Form processing




                  58
Form processing

• You want to redirect after successful
  submission.




                                          58
Form processing

• You want to redirect after successful
  submission.
• Supply a default URL.




                                          58
Form processing

• You want to redirect after successful
  submission.
• Supply a default URL.
 • (Preferably by using reverse resolution).




                                               58
Form processing

• You want to redirect after successful
  submission.
• Supply a default URL.
  • (Preferably by using reverse resolution).
• Let users override the default.



                                                58
def edit_entry(request, entry_id):
  form = EntryForm(request.POST or None)
  if form.is_valid():
      form.save()
      return redirect('entry_detail', entry_id)
  return render_to_response('entry/form.html', {…})




                                                      59
def edit_entry(request, entry_id,
          form_class=EntryForm,
          template_name='entry/form.html',
          post_save_redirect=None):

  form = form_class(request.POST or None)
  if form.is_valid():
      form.save()
      if post_save_redirect:
         return redirect(post_save_redirect)
      else:
         return redirect('entry_detail', entry_id)

  return render_to_response([template_name, 'entry/form.html'], {…})




                                                                       60
URLs




       61
URLs


• Provide a URLConf with all views.




                                      61
URLs


• Provide a URLConf with all views.
• Use named URL patterns.




                                      61
URLs


• Provide a URLConf with all views.
• Use named URL patterns.
• Use reverse lookups (by name).




                                      61
4.
Build to distribute (even private code).




                                           62
What the tutorial teaches

myproject/
 settings.py
 urls.py

  myapp/
   models.py

  mysecondapp/
   views.py

  …



                          63
from myproject.myapp.models import …
from myproject. myapp.models import …

…

myproject.settings
myproject.urls




                                        64
Project coupling
  kills re-use



                   65
Projects in real life.




                         66
Projects in real life.

• A settings module.




                         66
Projects in real life.

• A settings module.
• A root URLConf.




                         66
Projects in real life.

• A settings module.
• A root URLConf.
• Maybe a manage.py (but…)




                             66
Projects in real life.

• A settings module.
• A root URLConf.
• Maybe a manage.py (but…)
• And that’s it.



                             66
Advantages




             67
Advantages

• No assumptions about where things live.




                                            67
Advantages

• No assumptions about where things live.
• No PYTHONPATH magic.




                                            67
Advantages

• No assumptions about where things live.
• No PYTHONPATH magic.
• Reminds you that “projects” are just a
  Python module.



                                            67
You don’t even need a project




                            68
ljworld.com:




               69
ljworld.com:


• worldonline.settings.ljworld




                                 69
ljworld.com:


• worldonline.settings.ljworld
• worldonline.urls.ljworld




                                 69
ljworld.com:


• worldonline.settings.ljworld
• worldonline.urls.ljworld
• And a whole bunch of apps.




                                 69
Where apps really live




                         70
Where apps really live
• Single module directly on Python path
  (registration, tagging, etc.).




                                          70
Where apps really live
• Single module directly on Python path
  (registration, tagging, etc.).
• Related modules under a top-level
  package (ellington.events,
  ellington.podcasts, etc.)




                                          70
Where apps really live
• Single module directly on Python path
  (registration, tagging, etc.).
• Related modules under a top-level
  package (ellington.events,
  ellington.podcasts, etc.)
• No projects (ellington.settings doesn’t
  exist).


                                            70
Want to distribute?




                      71
Want to distribute?

• Build a package with distutils/setuptools.




                                               71
Want to distribute?

• Build a package with distutils/setuptools.
• Put it on PyPI (or a private package
  server).




                                               71
Want to distribute?

• Build a package with distutils/setuptools.
• Put it on PyPI (or a private package
  server).
• Now it works with easy_install, pip,
  buildout, …



                                               71
General best practices




                         72
General best practices
• Establish dependancy rules.




                                72
General best practices
• Establish dependancy rules.
• Establish a minimum Python version
  (suggestion: Python 2.5).




                                       72
General best practices
• Establish dependancy rules.
• Establish a minimum Python version
  (suggestion: Python 2.5).
• Establish a minimum Django version
  (suggestion: Django 1.0).




                                       72
General best practices
• Establish dependancy rules.
• Establish a minimum Python version
  (suggestion: Python 2.5).
• Establish a minimum Django version
  (suggestion: Django 1.0).
• Test frequently against new versions
  of dependancies.

                                         72
Document obsessively.



                    73
5.
Embrace and extend.




                      74
Don’t touch!




               75
Don’t touch!

• Good applications are extensible
  without patching.




                                     75
Don’t touch!

• Good applications are extensible
  without patching.
• Take advantage of every extensibility point
  an application gives you.




                                                75
Don’t touch!

• Good applications are extensible
  without patching.
• Take advantage of every extensibility point
  an application gives you.
• You may end up doing something that
  deserves a new application anyway.


                                                75
But this application
wasn’t meant to be
     extended!


                       76
Python Power!



                77
Extending a view




                   78
Extending a view


• Wrap the view with your own code.




                                      78
Extending a view


• Wrap the view with your own code.
• Doing it repetitively? Write a decorator.




                                              78
Extending a model




                    79
Extending a model


• Relate other models to it.




                               79
Extending a model


• Relate other models to it.
• Subclass it.




                               79
Extending a model


• Relate other models to it.
• Subclass it.
• Proxy subclasses (Django 1.1).




                                   79
Extending a form




                   80
Extending a form


• Subclass it.




                   80
Extending a form


• Subclass it.
• There is no step 2.




                        80
Other tricks




               81
Other tricks

• Signals lets you fire off customized
  behavior when certain events happen.




                                         81
Other tricks

• Signals lets you fire off customized
  behavior when certain events happen.
• Middleware offers full control over
  request/response handling.




                                         81
Other tricks

• Signals lets you fire off customized
  behavior when certain events happen.
• Middleware offers full control over
  request/response handling.
• Context processors can make additional
  information available if a view doesn’t.


                                             81
If you must make
    changes to
 external code…


                   82
Keep changes to a minimum




                            83
Keep changes to a minimum


• If possible, instead of adding a feature,
  add extensibility.




                                              83
Keep changes to a minimum


• If possible, instead of adding a feature,
  add extensibility.
• Keep as much changed code as you can
  out of the original app.




                                              83
Stay up-to-date




                  84
Stay up-to-date

• Don’t want to get out of sync with the
  original version of the code!




                                           84
Stay up-to-date

• Don’t want to get out of sync with the
  original version of the code!
• You might miss bugfixes.




                                           84
Stay up-to-date

• Don’t want to get out of sync with the
  original version of the code!
• You might miss bugfixes.
• You might even miss the feature you
  needed.



                                           84
Use a good VCS




                 85
Use a good VCS
• Subversion vendor branches don’t cut it.




                                             85
Use a good VCS
• Subversion vendor branches don’t cut it.
• DVCSes are perfect for this:




                                             85
Use a good VCS
• Subversion vendor branches don’t cut it.
• DVCSes are perfect for this:
 • Mercurial queues.




                                             85
Use a good VCS
• Subversion vendor branches don’t cut it.
• DVCSes are perfect for this:
 • Mercurial queues.
 • Git rebasing.




                                             85
Use a good VCS
• Subversion vendor branches don’t cut it.
• DVCSes are perfect for this:
 • Mercurial queues.
 • Git rebasing.
• At the very least, maintain a patch queue
  by hand.


                                              85
Be a good citizen




                    86
Be a good citizen

• If you change someone else’s code, let
  them know.




                                           86
Be a good citizen

• If you change someone else’s code, let
  them know.
• Maybe they’ll merge your changes in and
  you won’t have to fork anymore.




                                            86
Further reading




                  87
Deployment



             88
Deployment should...




                       89
Deployment should...
• Be automated.




                       89
Deployment should...
• Be automated.
• Automatically manage dependencies.




                                       89
Deployment should...
• Be automated.
• Automatically manage dependencies.
• Be isolated.




                                       89
Deployment should...
• Be automated.
• Automatically manage dependencies.
• Be isolated.
• Be repeatable.




                                       89
Deployment should...
• Be automated.
• Automatically manage dependencies.
• Be isolated.
• Be repeatable.
• Be identical in staging and in production.



                                               89
Deployment should...
• Be automated.
• Automatically manage dependencies.
• Be isolated.
• Be repeatable.
• Be identical in staging and in production.
• Work the same for everyone.

                                               89
Dependency
             Isolation   Automation
management




                                      90
Dependency
             Isolation   Automation
management




                                      90
Dependency
               Isolation      Automation
management

 apt/yum/...   virtualenv     Capistrano


easy_install   zc.buildout      Fabric


    pip                      Puppet/Chef/…


 zc.buildout

                                             90
Dependency
               Isolation      Automation
management

 apt/yum/...   virtualenv     Capistrano


easy_install   zc.buildout      Fabric


    pip                      Puppet/Chef/…


 zc.buildout

                                             90
Dependency
               Isolation      Automation
management

 apt/yum/...   virtualenv     Capistrano


easy_install   zc.buildout      Fabric


    pip                      Puppet/Chef/…


 zc.buildout

                                             90
Dependancy management




                    91
Dependancy management

• The Python ecosystem rocks!




                                91
Dependancy management

• The Python ecosystem rocks!
• Python package management doesn’t.




                                       91
Dependancy management

• The Python ecosystem rocks!
• Python package management doesn’t.
• Installing packages — and dependancies
  — correctly is a lot harder than it should be;
  most defaults are wrong.




                                               91
Dependancy management

• The Python ecosystem rocks!
• Python package management doesn’t.
• Installing packages — and dependancies
  — correctly is a lot harder than it should be;
  most defaults are wrong.
• Here be dragons.


                                               91
Vendor packages




                  92
Vendor packages

• APT, Yum, …




                  92
Vendor packages

• APT, Yum, …
• The good: familiar tools; stability; handles
  dependancies not on PyPI.




                                                 92
Vendor packages

• APT, Yum, …
• The good: familiar tools; stability; handles
  dependancies not on PyPI.
• The bad: small selection; not (very)
  portable; hard to supply user packages.




                                                 92
Vendor packages

• APT, Yum, …
• The good: familiar tools; stability; handles
  dependancies not on PyPI.
• The bad: small selection; not (very)
  portable; hard to supply user packages.
• The ugly: installs packages system-wide.


                                                 92
easy_install




               93
easy_install

• The good: multi-version packages.




                                      93
easy_install

• The good: multi-version packages.
• The bad: requires ‘net connection; can’t
  uninstall; can’t handle non-PyPI packages;
  multi-version packages barely work.




                                               93
easy_install

• The good: multi-version packages.
• The bad: requires ‘net connection; can’t
  uninstall; can’t handle non-PyPI packages;
  multi-version packages barely work.
• The ugly: stale; unsupported; defaults
  almost totally wrong; installs system-wide.



                                                93
pip
http://pip.openplans.org/




                            94
pip
http://pip.openplans.org/

 • “Pip Installs Packages”




                             94
pip
http://pip.openplans.org/

 • “Pip Installs Packages”
 • The good: Just Works™; handles non-
   PyPI packages (including direct from
   SCM); repeatable dependancies;
   integrates with virtualenv for isolation.




                                               94
pip
http://pip.openplans.org/

 • “Pip Installs Packages”
 • The good: Just Works™; handles non-
   PyPI packages (including direct from
   SCM); repeatable dependancies;
   integrates with virtualenv for isolation.
 • The bad: still young; not yet bundled.



                                               94
pip
http://pip.openplans.org/

 • “Pip Installs Packages”
 • The good: Just Works™; handles non-
   PyPI packages (including direct from
   SCM); repeatable dependancies;
   integrates with virtualenv for isolation.
 • The bad: still young; not yet bundled.
 • The ugly: haven’t found it yet.

                                               94
zc.buildout
http://buildout.org/




                       95
zc.buildout
http://buildout.org/

 • The good: incredibly flexible; handles any
   sort of dependancy; repeatable builds;
   reusable “recipes;” good ecosystem;
   handles isolation, too.




                                               95
zc.buildout
http://buildout.org/

 • The good: incredibly flexible; handles any
   sort of dependancy; repeatable builds;
   reusable “recipes;” good ecosystem;
   handles isolation, too.
 • The bad: often cryptic, INI-style
   configuration file; confusing duplication of
   recipes; sometimes too flexible.



                                                95
zc.buildout
http://buildout.org/

 • The good: incredibly flexible; handles any
   sort of dependancy; repeatable builds;
   reusable “recipes;” good ecosystem;
   handles isolation, too.
 • The bad: often cryptic, INI-style
   configuration file; confusing duplication of
   recipes; sometimes too flexible.
 • The ugly: nearly completely undocumented.

                                                95
Package isolation




                    96
Package isolation

• Why?




                    96
Package isolation

• Why?
 • Site A requires Foo v1.0; site B requires
   Foo v2.0.




                                               96
Package isolation

• Why?
 • Site A requires Foo v1.0; site B requires
   Foo v2.0.
 • You need to develop against multiple
   versions of dependancies.



                                               96
Package isolation tools




                          97
Package isolation tools
• Virtual machines (Xen, VMWare, EC2, …)




                                           97
Package isolation tools
• Virtual machines (Xen, VMWare, EC2, …)
• Multiple Python installations.




                                           97
Package isolation tools
• Virtual machines (Xen, VMWare, EC2, …)
• Multiple Python installations.
• “Virtual” Python installations.




                                           97
Package isolation tools
• Virtual machines (Xen, VMWare, EC2, …)
• Multiple Python installations.
• “Virtual” Python installations.
  • virtualenv
   http://pypi.python.org/pypi/virtualenv




                                            97
Package isolation tools
• Virtual machines (Xen, VMWare, EC2, …)
• Multiple Python installations.
• “Virtual” Python installations.
  • virtualenv
   http://pypi.python.org/pypi/virtualenv

  • zc.buildout
   http://buildout.org/


                                            97
Why automate?




                98
Why automate?
• “I can’t push this fix to the servers until
  Alex gets back from lunch.”




                                               98
Why automate?
• “I can’t push this fix to the servers until
  Alex gets back from lunch.”
• “Sorry, I can’t fix that. I’m new here.”




                                               98
Why automate?
• “I can’t push this fix to the servers until
  Alex gets back from lunch.”
• “Sorry, I can’t fix that. I’m new here.”
• “Oops, I just made the wrong version of
  our site live.”




                                               98
Why automate?
• “I can’t push this fix to the servers until
  Alex gets back from lunch.”
• “Sorry, I can’t fix that. I’m new here.”
• “Oops, I just made the wrong version of
  our site live.”
• “It’s broken! What’d you do!?”


                                               98
Automation basics




                    99
Automation basics


• SSH is right out.




                      99
Automation basics


• SSH is right out.
• Don’t futz with the server. Write a recipe.




                                                99
Automation basics


• SSH is right out.
• Don’t futz with the server. Write a recipe.
• Deploys should be idempotent.




                                                99
Capistrano
http://capify.org/




                     100
Capistrano
http://capify.org/



 • The good: lots of features; good
   documentation; active community.




                                      100
Capistrano
http://capify.org/



 • The good: lots of features; good
   documentation; active community.
 • The bad: stale development; very
   “opinionated” and Rails-oriented.




                                       100
Fabric
http://fabfile.org/




                     101
Fabric
http://fabfile.org/




 • The good: very simple; flexible; actively
   developed; Python.




                                              101
Fabric
http://fabfile.org/




 • The good: very simple; flexible; actively
   developed; Python.
 • The bad: no high-level commands; in flux.




                                              101
Configuration management




                          102
Configuration management


• CFEngine, Puppet, Chef, …




                              102
Configuration management


• CFEngine, Puppet, Chef, …
• Will handle a lot more than code
  deployment!




                                     102
Configuration management


• CFEngine, Puppet, Chef, …
• Will handle a lot more than code
  deployment!
• I only know a little about these.



                                      102
Recommendations
  Pip, Virtualenv, and Fabric

  Buildout and Fabric.

  Buildout and Puppet/Chef/….

  Utility computing and Puppet/Chef/….


                                         103
Production
environments


               104
net.
                            LiveJournal Backend: Today
                                                           (Roughly.)

    BIG-IP
                            perlbal (httpd/proxy)                                           Global Database
            bigip1                                        mod_perl
            bigip2                  proxy1                                                       master_a master_b
                                                            web1
                                    proxy2                  web2
                                    proxy3                  web3                Memcached
                                                                                            slave1 slave2     ...   slave5
  djabberd                          proxy4                                         mc1
                                                            web4
      djabberd                      proxy5
                                                              ...                  mc2          User DB Cluster 1
      djabberd
                                                            webN                   mc3             uc1a         uc1b
                                                                                   mc4          User DB Cluster 2
                                                                                    ...            uc2a         uc2b
                                                    gearmand
 Mogile Storage Nodes                                   gearmand1                  mcN          User DB Cluster 3
     sto1            sto2                               gearmandN                                  uc3a         uc3b
                                Mogile Trackers
      ...            sto8
                                 tracker1    tracker3                                           User DB Cluster N
                                                                                                   ucNa         ucNb
    MogileFS Database                                               “workers”
                                                                       gearwrkN                 Job Queues (xN)
        mog_a           mog_b                                         theschwkN                    jqNa         jqNb


     slave1     slaveN
http://danga.com/words/
                                 Brad Fitzpatrik, http://danga.com/words/2007_06_usenix/
                                                                                                                             3
                                                                                                                                 105
django

database

 media
 server




           106
Application servers




                      107
Application servers

• Apache + mod_python




                        107
Application servers

• Apache + mod_python
• Apache + mod_wsgi




                        107
Application servers

• Apache + mod_python
• Apache + mod_wsgi
• Apache/lighttpd + FastCGI




                              107
Application servers

• Apache + mod_python
• Apache + mod_wsgi
• Apache/lighttpd + FastCGI
• SCGI, AJP, nginx/mod_wsgi, ...



                                   107
Use mod_wsgi



               108
WSGIScriptAlias / /home/mysite/mysite.wsgi




                                             109
import os, sys

# Add to PYTHONPATH whatever you need
sys.path.append('/usr/local/django')

# Set DJANGO_SETTINGS_MODULE
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'

# Create the application for mod_wsgi
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()




                                                           110
“Scale”



          111
Does this scale?

       django

      database

       media
       server




                   112
Does this scale?

       django

      database

       media
       server




    Maybe!
                   112
Things per secong




                    Number of things


                                       113
Real-world example
      Database A

      175 req/s


      Database B

       75 req/s


                     114
Real-world example




    http://tweakers.net/reviews/657/6
                                        115
django

    media
  web server



  database

database server




                  116
Why separate hardware?




                     117
Why separate hardware?


• Resource contention




                        117
Why separate hardware?


• Resource contention
• Separate performance concerns




                                  117
Why separate hardware?


• Resource contention
• Separate performance concerns
• 0 → 1 is much harder than 1 → N




                                    117
DATABASE_HOST = '10.0.0.100'




                               118
DATABASE_HOST = '10.0.0.100'




                 FAIL          118
Connection middleware




                        119
Connection middleware
• Proxy between web and database layers




                                          119
Connection middleware
• Proxy between web and database layers
• Most implement hot fallover and
  connection pooling




                                          119
Connection middleware
• Proxy between web and database layers
• Most implement hot fallover and
  connection pooling
 • Some also provide replication, load
   balancing, parallel queries, connection
   limiting, &c



                                             119
Connection middleware
• Proxy between web and database layers
• Most implement hot fallover and
  connection pooling
 • Some also provide replication, load
   balancing, parallel queries, connection
   limiting, &c
• DATABASE_HOST = '127.0.0.1'

                                             119
Connection middleware




                        120
Connection middleware

• PostgreSQL: pgpool




                        120
Connection middleware

• PostgreSQL: pgpool
• MySQL: MySQL Proxy




                        120
Connection middleware

• PostgreSQL: pgpool
• MySQL: MySQL Proxy
• Database-agnostic: sqlrelay




                                120
Connection middleware

• PostgreSQL: pgpool
• MySQL: MySQL Proxy
• Database-agnostic: sqlrelay
• Oracle: ?



                                120
django           media
  web server      media server




  database
database server




                                 121
Media server traits




                      122
Media server traits

• Fast




                      122
Media server traits

• Fast
• Lightweight




                      122
Media server traits

• Fast
• Lightweight
• Optimized for high concurrency




                                   122
Media server traits

• Fast
• Lightweight
• Optimized for high concurrency
• Low memory overhead




                                   122
Media server traits

• Fast
• Lightweight
• Optimized for high concurrency
• Low memory overhead
• Good HTTP citizen


                                   122
Media servers




                123
Media servers

• Apache?




                123
Media servers

• Apache?
• lighttpd




                123
Media servers

• Apache?
• lighttpd
• nginx




                123
Media servers

• Apache?
• lighttpd
• nginx
• S3



                123
The absolute minimum


        django           media
       web server      media server




       database
     database server




                                      124
The absolute minimum


      django                 media




     database


                web server




                                     125
proxy                media

           load balancer         media server




django        django          django
         web server cluster




            database
          database server


                                                126
Why load balancers?



                      127
Load balancer traits




                       128
Load balancer traits

• Low memory overhead




                        128
Load balancer traits

• Low memory overhead
• High concurrency




                        128
Load balancer traits

• Low memory overhead
• High concurrency
• Hot fallover




                        128
Load balancer traits

• Low memory overhead
• High concurrency
• Hot fallover
• Other nifty features...



                            128
Load balancers




                 129
Load balancers

• Apache + mod_proxy




                       129
Load balancers

• Apache + mod_proxy
• perlbal




                       129
Load balancers

• Apache + mod_proxy
• perlbal
• nginx




                       129
Load balancers

• Apache + mod_proxy
• perlbal
• nginx
• Varnish




                       129
Load balancers

• Apache + mod_proxy
• perlbal
• nginx
• Varnish
• Squid


                       129
CREATE POOL mypool
  POOL mypool ADD 10.0.0.100
  POOL mypool ADD 10.0.0.101

CREATE SERVICE mysite
  SET listen = my.public.ip
  SET role = reverse_proxy
  SET pool = mypool
  SET verify_backend = on
  SET buffer_size = 120k
ENABLE mysite




                               130
you@yourserver:~$ telnet localhost 60000

pool mysite add 10.0.0.102
OK

nodes 10.0.0.101
10.0.0.101 lastresponse 1237987449
10.0.0.101 requests 97554563
10.0.0.101 connects 129242435
10.0.0.101 lastconnect 1237987449
10.0.0.101 attempts 129244743
10.0.0.101 responsecodes 200 358
10.0.0.101 responsecodes 302 14
10.0.0.101 responsecodes 207 99
10.0.0.101 responsecodes 301 11
10.0.0.101 responsecodes 404 18
10.0.0.101 lastattempt 1237987449



                                           131
proxy             proxy              proxy     media           media

           load balancing cluster                media server cluster




 django           django              django    cache           cache
             web server cluster                     cache cluster




database         database            database

           database server cluster



                                                                        132
“Shared nothing”



                   133
BALANCE = None

def balance_sheet(request):
  global BALANCE
  if not BALANCE:
      bank = Bank.objects.get(...)
      BALANCE = bank.total_balance()
  ...




                                       134
BALANCE = None

def balance_sheet(request):
  global BALANCE
  if not BALANCE:
      bank = Bank.objects.get(...)
      BALANCE = bank.total_balance()
  ...




                   FAIL                134
Global variables are
     right out


                       135
from django.cache import cache

def balance_sheet(request):
  balance = cache.get('bank_balance')
  if not balance:
     bank = Bank.objects.get(...)
     balance = bank.total_balance()
      cache.set('bank_balance', balance)

  ...




                                           136
from django.cache import cache

def balance_sheet(request):
  balance = cache.get('bank_balance')
  if not balance:
     bank = Bank.objects.get(...)
     balance = bank.total_balance()
      cache.set('bank_balance', balance)

  ...




                     WIN                   136
def generate_report(request):
  report = get_the_report()
  open('/tmp/report.txt', 'w').write(report)
  return redirect(view_report)

def view_report(request):
  report = open('/tmp/report.txt').read()
  return HttpResponse(report)




                                               137
def generate_report(request):
  report = get_the_report()
  open('/tmp/report.txt', 'w').write(report)
  return redirect(view_report)

def view_report(request):
  report = open('/tmp/report.txt').read()
  return HttpResponse(report)




                    FAIL                       137
Filesystem?
What filesystem?


                  138
Further reading




                  139
Further reading

• Cal Henderson, Building Scalable Web Sites




                                               139
Further reading

• Cal Henderson, Building Scalable Web Sites
• John Allspaw, The Art of Capacity Planning




                                               139
Further reading

• Cal Henderson, Building Scalable Web Sites
• John Allspaw, The Art of Capacity Planning
• http://kitchensoap.com/




                                               139
Further reading

• Cal Henderson, Building Scalable Web Sites
• John Allspaw, The Art of Capacity Planning
• http://kitchensoap.com/
• http://highscalability.com/




                                               139
Monitoring



             140
Goals




        141
Goals
• When the site goes down, know it immediately.




                                                  141
Goals
• When the site goes down, know it immediately.
• Automatically handle common sources of
  downtime.




                                                  141
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World
Django In The Real World

Contenu connexe

Tendances

Android Custom Views
Android Custom ViewsAndroid Custom Views
Android Custom ViewsBabar Sanah
 
Functional programming with Java 8
Functional programming with Java 8Functional programming with Java 8
Functional programming with Java 8LivePerson
 
Workshop 4: NodeJS. Express Framework & MongoDB.
Workshop 4: NodeJS. Express Framework & MongoDB.Workshop 4: NodeJS. Express Framework & MongoDB.
Workshop 4: NodeJS. Express Framework & MongoDB.Visual Engineering
 
Développement d'applications pour la plateforme Java EE
Développement d'applications pour la plateforme Java EEDéveloppement d'applications pour la plateforme Java EE
Développement d'applications pour la plateforme Java EESabri Bouchlema
 
Web Development with Python and Django
Web Development with Python and DjangoWeb Development with Python and Django
Web Development with Python and DjangoMichael Pirnat
 
Exception Handling In Java
Exception Handling In JavaException Handling In Java
Exception Handling In Javaparag
 
Java 8 - collections et stream
Java 8 - collections et streamJava 8 - collections et stream
Java 8 - collections et streamFranck SIMON
 
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...Edureka!
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.jsVikash Singh
 
Mohamed youssfi support architectures logicielles distribuées basées sue les ...
Mohamed youssfi support architectures logicielles distribuées basées sue les ...Mohamed youssfi support architectures logicielles distribuées basées sue les ...
Mohamed youssfi support architectures logicielles distribuées basées sue les ...ENSET, Université Hassan II Casablanca
 

Tendances (20)

Android Custom Views
Android Custom ViewsAndroid Custom Views
Android Custom Views
 
Django Girls Tutorial
Django Girls TutorialDjango Girls Tutorial
Django Girls Tutorial
 
Functional programming with Java 8
Functional programming with Java 8Functional programming with Java 8
Functional programming with Java 8
 
Workshop 4: NodeJS. Express Framework & MongoDB.
Workshop 4: NodeJS. Express Framework & MongoDB.Workshop 4: NodeJS. Express Framework & MongoDB.
Workshop 4: NodeJS. Express Framework & MongoDB.
 
Theses Soutenues sous Direction et Co-Direction du Pr YOUSSFI
Theses Soutenues sous Direction et Co-Direction du Pr YOUSSFITheses Soutenues sous Direction et Co-Direction du Pr YOUSSFI
Theses Soutenues sous Direction et Co-Direction du Pr YOUSSFI
 
Java beans
Java beansJava beans
Java beans
 
Développement d'applications pour la plateforme Java EE
Développement d'applications pour la plateforme Java EEDéveloppement d'applications pour la plateforme Java EE
Développement d'applications pour la plateforme Java EE
 
Web Development with Python and Django
Web Development with Python and DjangoWeb Development with Python and Django
Web Development with Python and Django
 
Exception Handling In Java
Exception Handling In JavaException Handling In Java
Exception Handling In Java
 
Introduction to thymeleaf
Introduction to thymeleafIntroduction to thymeleaf
Introduction to thymeleaf
 
Java & advanced java
Java & advanced javaJava & advanced java
Java & advanced java
 
Mise en oeuvre des framework de machines et deep learning v1
Mise en oeuvre des framework de machines et deep learning v1 Mise en oeuvre des framework de machines et deep learning v1
Mise en oeuvre des framework de machines et deep learning v1
 
Var arg methods
Var arg methodsVar arg methods
Var arg methods
 
ReactJS presentation.pptx
ReactJS presentation.pptxReactJS presentation.pptx
ReactJS presentation.pptx
 
Polymorphisme
PolymorphismePolymorphisme
Polymorphisme
 
Java 8 - collections et stream
Java 8 - collections et streamJava 8 - collections et stream
Java 8 - collections et stream
 
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...
Django Rest Framework | How to Create a RESTful API Using Django | Django Tut...
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
Mohamed youssfi support architectures logicielles distribuées basées sue les ...
Mohamed youssfi support architectures logicielles distribuées basées sue les ...Mohamed youssfi support architectures logicielles distribuées basées sue les ...
Mohamed youssfi support architectures logicielles distribuées basées sue les ...
 
Reactjs
ReactjsReactjs
Reactjs
 

En vedette

Two scoops of Django - Security Best Practices
Two scoops of Django - Security Best PracticesTwo scoops of Django - Security Best Practices
Two scoops of Django - Security Best PracticesSpin Lai
 
Getting Started With Django
Getting Started With DjangoGetting Started With Django
Getting Started With Djangojeff_croft
 
Blog project
Blog projectBlog project
Blog projectshara831
 
Starters with Django
Starters with Django Starters with Django
Starters with Django BeDjango
 
API Management Demystified
API Management DemystifiedAPI Management Demystified
API Management DemystifiedSoftware AG
 
12 tips on Django Best Practices
12 tips on Django Best Practices12 tips on Django Best Practices
12 tips on Django Best PracticesDavid Arcos
 
Scalable Django Architecture
Scalable Django ArchitectureScalable Django Architecture
Scalable Django ArchitectureRami Sayar
 

En vedette (10)

Two scoops of Django - Security Best Practices
Two scoops of Django - Security Best PracticesTwo scoops of Django - Security Best Practices
Two scoops of Django - Security Best Practices
 
Getting Started With Django
Getting Started With DjangoGetting Started With Django
Getting Started With Django
 
Django in the Real World
Django in the Real WorldDjango in the Real World
Django in the Real World
 
Django packages list
Django packages listDjango packages list
Django packages list
 
Blog project
Blog projectBlog project
Blog project
 
Blog Project
Blog ProjectBlog Project
Blog Project
 
Starters with Django
Starters with Django Starters with Django
Starters with Django
 
API Management Demystified
API Management DemystifiedAPI Management Demystified
API Management Demystified
 
12 tips on Django Best Practices
12 tips on Django Best Practices12 tips on Django Best Practices
12 tips on Django Best Practices
 
Scalable Django Architecture
Scalable Django ArchitectureScalable Django Architecture
Scalable Django Architecture
 

Similaire à Django In The Real World

Deep dive into azure virtual machines
Deep dive into azure virtual machinesDeep dive into azure virtual machines
Deep dive into azure virtual machinesJasjit Chopra
 
Productionizing Machine Learning with a Microservices Architecture
Productionizing Machine Learning with a Microservices ArchitectureProductionizing Machine Learning with a Microservices Architecture
Productionizing Machine Learning with a Microservices ArchitectureDatabricks
 
Server and application monitoring webinars [Applications Manager]: Part 1
Server and application monitoring webinars [Applications Manager]: Part 1Server and application monitoring webinars [Applications Manager]: Part 1
Server and application monitoring webinars [Applications Manager]: Part 1ManageEngine, Zoho Corporation
 
Enabling .NET Apps with Monitoring and Management Using Steeltoe
Enabling .NET Apps with Monitoring and Management Using SteeltoeEnabling .NET Apps with Monitoring and Management Using Steeltoe
Enabling .NET Apps with Monitoring and Management Using SteeltoeVMware Tanzu
 
Server and application monitoring webinars [Applications Manager] - Part 2
Server and application monitoring webinars [Applications Manager] - Part 2Server and application monitoring webinars [Applications Manager] - Part 2
Server and application monitoring webinars [Applications Manager] - Part 2ManageEngine, Zoho Corporation
 
Sutol How To Be A Lion Tamer
Sutol How To Be A Lion TamerSutol How To Be A Lion Tamer
Sutol How To Be A Lion TamerSharon James
 
Realtime streaming architecture in INFINARIO
Realtime streaming architecture in INFINARIORealtime streaming architecture in INFINARIO
Realtime streaming architecture in INFINARIOJozo Kovac
 
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014Piyush Kumar
 
Opensource approach to design and deployment of Microservices based VNF
Opensource approach to design and deployment of Microservices based VNFOpensource approach to design and deployment of Microservices based VNF
Opensource approach to design and deployment of Microservices based VNFMichelle Holley
 
What's New in IBM Streams V4.1
What's New in IBM Streams V4.1What's New in IBM Streams V4.1
What's New in IBM Streams V4.1lisanl
 
Shared Security Responsibility for the Azure Cloud
Shared Security Responsibility for the Azure CloudShared Security Responsibility for the Azure Cloud
Shared Security Responsibility for the Azure CloudAlert Logic
 
Common Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the CloudCommon Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the CloudNick Gerner
 
How to be a lion tamer
How to be a lion tamerHow to be a lion tamer
How to be a lion tamerWannes Rams
 
How to be a lion tamer
How to be a lion tamerHow to be a lion tamer
How to be a lion tamerSharon James
 
Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!DataWorks Summit
 
20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers SoftwareDevOps Chicago
 
Docker based Architecture by Denys Serdiuk
Docker based Architecture by Denys SerdiukDocker based Architecture by Denys Serdiuk
Docker based Architecture by Denys SerdiukLohika_Odessa_TechTalks
 

Similaire à Django In The Real World (20)

HTML5 Security
HTML5 SecurityHTML5 Security
HTML5 Security
 
Deep dive into azure virtual machines
Deep dive into azure virtual machinesDeep dive into azure virtual machines
Deep dive into azure virtual machines
 
Productionizing Machine Learning with a Microservices Architecture
Productionizing Machine Learning with a Microservices ArchitectureProductionizing Machine Learning with a Microservices Architecture
Productionizing Machine Learning with a Microservices Architecture
 
Server and application monitoring webinars [Applications Manager]: Part 1
Server and application monitoring webinars [Applications Manager]: Part 1Server and application monitoring webinars [Applications Manager]: Part 1
Server and application monitoring webinars [Applications Manager]: Part 1
 
Enabling .NET Apps with Monitoring and Management Using Steeltoe
Enabling .NET Apps with Monitoring and Management Using SteeltoeEnabling .NET Apps with Monitoring and Management Using Steeltoe
Enabling .NET Apps with Monitoring and Management Using Steeltoe
 
Server and application monitoring webinars [Applications Manager] - Part 2
Server and application monitoring webinars [Applications Manager] - Part 2Server and application monitoring webinars [Applications Manager] - Part 2
Server and application monitoring webinars [Applications Manager] - Part 2
 
Sutol How To Be A Lion Tamer
Sutol How To Be A Lion TamerSutol How To Be A Lion Tamer
Sutol How To Be A Lion Tamer
 
Realtime streaming architecture in INFINARIO
Realtime streaming architecture in INFINARIORealtime streaming architecture in INFINARIO
Realtime streaming architecture in INFINARIO
 
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014
 
Monitoring in 2017 - TIAD Camp Docker
Monitoring in 2017 - TIAD Camp DockerMonitoring in 2017 - TIAD Camp Docker
Monitoring in 2017 - TIAD Camp Docker
 
Opensource approach to design and deployment of Microservices based VNF
Opensource approach to design and deployment of Microservices based VNFOpensource approach to design and deployment of Microservices based VNF
Opensource approach to design and deployment of Microservices based VNF
 
What's New in IBM Streams V4.1
What's New in IBM Streams V4.1What's New in IBM Streams V4.1
What's New in IBM Streams V4.1
 
Microservices
MicroservicesMicroservices
Microservices
 
Shared Security Responsibility for the Azure Cloud
Shared Security Responsibility for the Azure CloudShared Security Responsibility for the Azure Cloud
Shared Security Responsibility for the Azure Cloud
 
Common Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the CloudCommon Sense Performance Indicators in the Cloud
Common Sense Performance Indicators in the Cloud
 
How to be a lion tamer
How to be a lion tamerHow to be a lion tamer
How to be a lion tamer
 
How to be a lion tamer
How to be a lion tamerHow to be a lion tamer
How to be a lion tamer
 
Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!Analyzing the World's Largest Security Data Lake!
Analyzing the World's Largest Security Data Lake!
 
20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software20140708 - Jeremy Edberg: How Netflix Delivers Software
20140708 - Jeremy Edberg: How Netflix Delivers Software
 
Docker based Architecture by Denys Serdiuk
Docker based Architecture by Denys SerdiukDocker based Architecture by Denys Serdiuk
Docker based Architecture by Denys Serdiuk
 

Plus de Jacob Kaplan-Moss

Introduction To Django (Strange Loop 2011)
Introduction To Django (Strange Loop 2011)Introduction To Django (Strange Loop 2011)
Introduction To Django (Strange Loop 2011)Jacob Kaplan-Moss
 
The Best (and Worst) of Django
The Best (and Worst) of DjangoThe Best (and Worst) of Django
The Best (and Worst) of DjangoJacob Kaplan-Moss
 
Writing great documentation - CodeConf 2011
Writing great documentation - CodeConf 2011Writing great documentation - CodeConf 2011
Writing great documentation - CodeConf 2011Jacob Kaplan-Moss
 
Django Introduction, Dev in Rio 2009
Django Introduction, Dev in Rio 2009Django Introduction, Dev in Rio 2009
Django Introduction, Dev in Rio 2009Jacob Kaplan-Moss
 
Building a web framework: Django's design decisions
Building a web framework: Django's design decisionsBuilding a web framework: Django's design decisions
Building a web framework: Django's design decisionsJacob Kaplan-Moss
 
Django - the first five years
Django - the first five yearsDjango - the first five years
Django - the first five yearsJacob Kaplan-Moss
 
A brief history of Django model syntax
A brief history of Django model syntaxA brief history of Django model syntax
A brief history of Django model syntaxJacob Kaplan-Moss
 

Plus de Jacob Kaplan-Moss (11)

Introduction To Django (Strange Loop 2011)
Introduction To Django (Strange Loop 2011)Introduction To Django (Strange Loop 2011)
Introduction To Django (Strange Loop 2011)
 
The Best (and Worst) of Django
The Best (and Worst) of DjangoThe Best (and Worst) of Django
The Best (and Worst) of Django
 
Writing great documentation - CodeConf 2011
Writing great documentation - CodeConf 2011Writing great documentation - CodeConf 2011
Writing great documentation - CodeConf 2011
 
What's new in Django 1.2?
What's new in Django 1.2?What's new in Django 1.2?
What's new in Django 1.2?
 
Django Introduction, Dev in Rio 2009
Django Introduction, Dev in Rio 2009Django Introduction, Dev in Rio 2009
Django Introduction, Dev in Rio 2009
 
Snakes on the Web
Snakes on the WebSnakes on the Web
Snakes on the Web
 
Building a web framework: Django's design decisions
Building a web framework: Django's design decisionsBuilding a web framework: Django's design decisions
Building a web framework: Django's design decisions
 
State Of Django
State Of DjangoState Of Django
State Of Django
 
Django - the first five years
Django - the first five yearsDjango - the first five years
Django - the first five years
 
A brief history of Django model syntax
A brief history of Django model syntaxA brief history of Django model syntax
A brief history of Django model syntax
 
Django Update (OSCON 2007)
Django Update (OSCON 2007)Django Update (OSCON 2007)
Django Update (OSCON 2007)
 

Dernier

20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 

Dernier (20)

20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 

Django In The Real World

  • 1. Django in the Real World Jacob Kaplan-Moss OSCON 2009 http://jacobian.org/TN
  • 2. Jacob Kaplan-Moss http://jacobian.org / jacob@jacobian.org / @jacobian Lead Developer, Django Partner, Revolution Systems 2
  • 3. Shameless plug: http://revsys.com/ 3
  • 4. Hat tip: James Bennett (http://b-list.org) 4
  • 5. So you’ve written a Django site… 5
  • 8. API Metering http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 9. API Metering • Backups & Snapshots http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 10. API Metering • Backups & Snapshots • Counters http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 11. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 12. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 13. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 14. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 15. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 16. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 17. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 18. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 19. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 20. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 21. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 22. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 23. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 24. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 25. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 26. API Metering • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 27. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 28. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 29. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 30. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 31. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 32. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 33. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 34. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 35. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 36. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 37. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 38. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 39. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Ranging • Rolling deployments • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 40. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Ranging • Rolling deployments • Geo • Multiple versions (selective beta) • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 41. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Ranging • Rolling deployments • Geo • Multiple versions (selective beta) • Sharding • Bucket Testing • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 42. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Ranging • Rolling deployments • Geo • Multiple versions (selective beta) • Sharding • Bucket Testing • Smart Caching • Rollbacks • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 43. API Metering • Distributed Log storage, analysis • Backups & Snapshots • Graphing • Counters • HTTP Caching • Cloud/Cluster Management Tools • Input/Output Filtering • Instrumentation/Monitoring • Memory Caching • Failover • Non-relational Key Stores • Node addition/removal and hashing • Rate Limiting • Auto-scaling for cloud resources • Relational Storage • CSRF/XSS Protection • Queues • Data Retention/Archival • Rate Limiting • Deployment Tools • Real-time messaging (XMPP) • Multiple Devs, Staging, Prod • Search • Data model upgrades • Ranging • Rolling deployments • Geo • Multiple versions (selective beta) • Sharding • Bucket Testing • Smart Caching • Rollbacks • Dirty-table management • CDN Management • Distributed File Storage http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites 7
  • 46. The bare minimum: • Test. • Structure for deployment. 8
  • 47. The bare minimum: • Test. • Structure for deployment. • Use deployment tools. 8
  • 48. The bare minimum: • Test. • Structure for deployment. • Use deployment tools. • Design a production environment. 8
  • 49. The bare minimum: • Test. • Structure for deployment. • Use deployment tools. • Design a production environment. • Monitor. 8
  • 50. The bare minimum: • Test. • Structure for deployment. • Use deployment tools. • Design a production environment. • Monitor. • Tune. 8
  • 51. Testing 9
  • 52. Tests are the Programmer’s stone, transmuting fear into ” boredom. — Kent Beck 10
  • 54. I don’t do test driven development. I do stupidity driven testing… I wait until I do something stupid, and ” then write tests to avoid doing it again. — Titus Brown 12
  • 55. Whatever happens, don’t let your test suite break thinking, “I’ll go back and fix this later.” 13
  • 57. Unit testing unittest Functional/behavior testing Browser testing 14
  • 58. Unit testing unittest doctest Functional/behavior testing Browser testing 14
  • 59. Unit testing unittest doctest Functional/behavior testing django.test.Client, Twill Browser testing 14
  • 60. Unit testing unittest doctest Functional/behavior testing django.test.Client, Twill Browser testing Windmill, Selenium 14
  • 61. You need them all. 15
  • 63. Testing Django • Unit tests (unittest) 16
  • 64. Testing Django • Unit tests (unittest) • Doctests (doctest) 16
  • 65. Testing Django • Unit tests (unittest) • Doctests (doctest) • Fixtures 16
  • 66. Testing Django • Unit tests (unittest) • Doctests (doctest) • Fixtures • Test client 16
  • 67. Testing Django • Unit tests (unittest) • Doctests (doctest) • Fixtures • Test client • Email capture 16
  • 70. Unit tests • “Whitebox” testing • Verify the small functional units of your app 17
  • 71. Unit tests • “Whitebox” testing • Verify the small functional units of your app • Very fine-grained 17
  • 72. Unit tests • “Whitebox” testing • Verify the small functional units of your app • Very fine-grained • Familier to most programmers (JUnit, NUnit, etc.) 17
  • 73. Unit tests • “Whitebox” testing • Verify the small functional units of your app • Very fine-grained • Familier to most programmers (JUnit, NUnit, etc.) • Provided in Python by unittest 17
  • 77. django.test.TestCase • Fixtures. • Test client. • Email capture. 18
  • 78. django.test.TestCase • Fixtures. • Test client. • Email capture. • Database management. 18
  • 79. django.test.TestCase • Fixtures. • Test client. • Email capture. • Database management. • Slower than unittest.TestCase. 18
  • 80. class StoryAddViewTests(TestCase): fixtures = ['authtestdata', 'newsbudget_test_data'] urls = 'newsbudget.urls' def test_story_add_get(self): r = self.client.get('/budget/stories/add/') self.assertEqual(r.status_code, 200) … def test_story_add_post(self): data = { 'title': 'Hungry cat is hungry', 'date': '2009-01-01', } r = self.client.post('/budget/stories/add/', data) self.assertEqual(r.status_code, 302) … 19
  • 81. Doctests 20
  • 82. Doctests • Easy to write & read. 20
  • 83. Doctests • Easy to write & read. • Produces self-documenting code. 20
  • 84. Doctests • Easy to write & read. • Produces self-documenting code. • Great for cases that only use assertEquals. 20
  • 85. Doctests • Easy to write & read. • Produces self-documenting code. • Great for cases that only use assertEquals. • Somewhere between unit tests and functional tests. 20
  • 86. Doctests • Easy to write & read. • Produces self-documenting code. • Great for cases that only use assertEquals. • Somewhere between unit tests and functional tests. • Difficult to debug. 20
  • 87. Doctests • Easy to write & read. • Produces self-documenting code. • Great for cases that only use assertEquals. • Somewhere between unit tests and functional tests. • Difficult to debug. • Don’t always provide useful test failures. 20
  • 88. class Choices(object): """ Easy declarative "choices" tool:: >>> STATUSES = Choices("Live", "Draft") # Acts like a choices list: >>> list(STATUSES) [(1, 'Live'), (2, 'Draft')] # Easily convert from code to verbose: >>> STATUSES.verbose(1) 'Live' # ... and vice versa: >>> STATUSES.code("Draft") 2 """ … 21
  • 89. **************************************************** File "utils.py", line 150, in __main__.Choices Failed example: STATUSES.verbose(1) Expected: 'Live' Got: 'Draft' **************************************************** 22
  • 91. Functional tests • a.k.a “Behavior Driven Development.” 23
  • 92. Functional tests • a.k.a “Behavior Driven Development.” • “Blackbox,” holistic testing. 23
  • 93. Functional tests • a.k.a “Behavior Driven Development.” • “Blackbox,” holistic testing. • All the hardcore TDD folks look down on functional tests. 23
  • 94. Functional tests • a.k.a “Behavior Driven Development.” • “Blackbox,” holistic testing. • All the hardcore TDD folks look down on functional tests. • But they keep your boss happy. 23
  • 95. Functional tests • a.k.a “Behavior Driven Development.” • “Blackbox,” holistic testing. • All the hardcore TDD folks look down on functional tests. • But they keep your boss happy. • Easy to find problems; harder to find the actual bug. 23
  • 102. django.test.Client • Test the whole request path without running a web server. 25
  • 103. django.test.Client • Test the whole request path without running a web server. • Responses provide extra information about templates and their contexts. 25
  • 104. class StoryAddViewTests(TestCase): fixtures = ['authtestdata', 'newsbudget_test_data'] urls = 'newsbudget.urls' def test_story_add_get(self): r = self.client.get('/budget/stories/add/') self.assertEqual(r.status_code, 200) … def test_story_add_post(self): data = { 'title': 'Hungry cat is hungry', 'date': '2009-01-01', } r = self.client.post('/budget/stories/add/', data) self.assertEqual(r.status_code, 302) … 26
  • 106. Web browser testing • The ultimate in functional testing for web applications. 27
  • 107. Web browser testing • The ultimate in functional testing for web applications. • Run test in a web browser. 27
  • 108. Web browser testing • The ultimate in functional testing for web applications. • Run test in a web browser. • Can verify JavaScript, AJAX; even CSS. 27
  • 109. Web browser testing • The ultimate in functional testing for web applications. • Run test in a web browser. • Can verify JavaScript, AJAX; even CSS. • Test your site across supported browsers. 27
  • 112. Browser testing tools • Selenium • Windmill 28
  • 114. “Exotic” testing • Static source analysis. 29
  • 115. “Exotic” testing • Static source analysis. • Smoke testing (crawlers and spiders). 29
  • 116. “Exotic” testing • Static source analysis. • Smoke testing (crawlers and spiders). • Monkey testing. 29
  • 117. “Exotic” testing • Static source analysis. • Smoke testing (crawlers and spiders). • Monkey testing. • Load testing. 29
  • 118. “Exotic” testing • Static source analysis. • Smoke testing (crawlers and spiders). • Monkey testing. • Load testing. • ... 29
  • 119. 30
  • 121. Further resources • Windmill talk here at OSCON http://bit.ly/14tkrd 31
  • 122. Further resources • Windmill talk here at OSCON http://bit.ly/14tkrd • Django testing documentation http://bit.ly/django-testing 31
  • 123. Further resources • Windmill talk here at OSCON http://bit.ly/14tkrd • Django testing documentation http://bit.ly/django-testing • Python Testing Tools Taxonomy http://bit.ly/py-testing-tools 31
  • 126. Designing for reuse • Do one thing, and do it well. 33
  • 127. Designing for reuse • Do one thing, and do it well. • Don’t be afraid of multiple apps. 33
  • 128. Designing for reuse • Do one thing, and do it well. • Don’t be afraid of multiple apps. • Write for flexibility. 33
  • 129. Designing for reuse • Do one thing, and do it well. • Don’t be afraid of multiple apps. • Write for flexibility. • Build to distribute. 33
  • 130. Designing for reuse • Do one thing, and do it well. • Don’t be afraid of multiple apps. • Write for flexibility. • Build to distribute. • Extend carefully. 33
  • 131. 1. Do one thing, and do it well. 34
  • 133. Focus 36
  • 134. Focus • Ask yourself: “What does this application do?” 36
  • 135. Focus • Ask yourself: “What does this application do?” • Answer should be one or two short sentences. 36
  • 136. Good focus 37
  • 137. Good focus • “Handle storage of users and authentication of their identities.” 37
  • 138. Good focus • “Handle storage of users and authentication of their identities.” • “Allow content to be tagged, del.icio.us style, with querying by tags.” 37
  • 139. Good focus • “Handle storage of users and authentication of their identities.” • “Allow content to be tagged, del.icio.us style, with querying by tags.” • “Handle entries in a weblog.” 37
  • 140. Bad focus 38
  • 141. Bad focus • “Handle entries in a weblog, and users who post them, and their authentication, and tagging and categorization, and some flat pages for static content, and...” 38
  • 143. Warning signs • Lots of files. 39
  • 144. Warning signs • Lots of files. • Lots of modules. 39
  • 145. Warning signs • Lots of files. • Lots of modules. • Lots of models. 39
  • 146. Warning signs • Lots of files. • Lots of modules. • Lots of models. • Lots of code. 39
  • 148. Small is good • Many great Django apps are very small. 40
  • 149. Small is good • Many great Django apps are very small. • Even a lot of “simple” Django sites commonly have a dozen or more applications in INSTALLED_APPS. 40
  • 150. Small is good • Many great Django apps are very small. • Even a lot of “simple” Django sites commonly have a dozen or more applications in INSTALLED_APPS. • If you’ve got a complex site and a short application list, something’s probably wrong. 40
  • 152. Approach features skeptically • What does the application do? 41
  • 153. Approach features skeptically • What does the application do? • Does this feature have anything to do with that? 41
  • 154. Approach features skeptically • What does the application do? • Does this feature have anything to do with that? • No? Don’t add it. 41
  • 155. 2. Don’t be afraid of many apps. 42
  • 157. The monolith anti-pattern • The “application” is the whole site. 43
  • 158. The monolith anti-pattern • The “application” is the whole site. • Re-use? YAGNI. 43
  • 159. The monolith anti-pattern • The “application” is the whole site. • Re-use? YAGNI. • Plugins that hook into the “main” application. 43
  • 160. The monolith anti-pattern • The “application” is the whole site. • Re-use? YAGNI. • Plugins that hook into the “main” application. • Heavy use of middleware-like concepts. 43
  • 163. The Django mindset • Application: some bit of functionality. 45
  • 164. The Django mindset • Application: some bit of functionality. • Site: several applications. 45
  • 165. The Django mindset • Application: some bit of functionality. • Site: several applications. • Spin off new “apps” liberally. 45
  • 166. The Django mindset • Application: some bit of functionality. • Site: several applications. • Spin off new “apps” liberally. • Develop a suite of apps ready for when they’re needed. 45
  • 168. Django encourages this • INSTALLED_APPS 46
  • 169. Django encourages this • INSTALLED_APPS • Applications are just Python packages, not some Django-specific “app” or “plugin.” 46
  • 170. Django encourages this • INSTALLED_APPS • Applications are just Python packages, not some Django-specific “app” or “plugin.” • Abstractions like django.contrib.sites make you think about this as you develop. 46
  • 171. Spin off a new app? 47
  • 172. Spin off a new app? • Is this feature unrelated to the app’s focus? 47
  • 173. Spin off a new app? • Is this feature unrelated to the app’s focus? • Is it orthogonal to the rest of the app? 47
  • 174. Spin off a new app? • Is this feature unrelated to the app’s focus? • Is it orthogonal to the rest of the app? • Will I need similar functionality again? 47
  • 175. The ideal: 48
  • 176. I need a contact form 49
  • 177. urlpatterns = ('', … (r'^contact/', include('contact_form.urls')), … ) 50
  • 180. But… what about… • Site A wants a contact form that just collects a message. 52
  • 181. But… what about… • Site A wants a contact form that just collects a message. • Site B’s marketing department wants a bunch of info. 52
  • 182. But… what about… • Site A wants a contact form that just collects a message. • Site B’s marketing department wants a bunch of info. • Site C wants to use Akismet to filter automated spam. 52
  • 183. 53
  • 184. 53
  • 187. Common sense • Sane defaults. 55
  • 188. Common sense • Sane defaults. • Easy overrides. 55
  • 189. Common sense • Sane defaults. • Easy overrides. • Don’t set anything in stone. 55
  • 190. Forms 56
  • 191. Forms • Supply a form class. 56
  • 192. Forms • Supply a form class. • Let users specify their own. 56
  • 193. Templates 57
  • 194. Templates • Specify a default template. 57
  • 195. Templates • Specify a default template. • Let users specify their own. 57
  • 197. Form processing • You want to redirect after successful submission. 58
  • 198. Form processing • You want to redirect after successful submission. • Supply a default URL. 58
  • 199. Form processing • You want to redirect after successful submission. • Supply a default URL. • (Preferably by using reverse resolution). 58
  • 200. Form processing • You want to redirect after successful submission. • Supply a default URL. • (Preferably by using reverse resolution). • Let users override the default. 58
  • 201. def edit_entry(request, entry_id): form = EntryForm(request.POST or None) if form.is_valid(): form.save() return redirect('entry_detail', entry_id) return render_to_response('entry/form.html', {…}) 59
  • 202. def edit_entry(request, entry_id, form_class=EntryForm, template_name='entry/form.html', post_save_redirect=None): form = form_class(request.POST or None) if form.is_valid(): form.save() if post_save_redirect: return redirect(post_save_redirect) else: return redirect('entry_detail', entry_id) return render_to_response([template_name, 'entry/form.html'], {…}) 60
  • 203. URLs 61
  • 204. URLs • Provide a URLConf with all views. 61
  • 205. URLs • Provide a URLConf with all views. • Use named URL patterns. 61
  • 206. URLs • Provide a URLConf with all views. • Use named URL patterns. • Use reverse lookups (by name). 61
  • 207. 4. Build to distribute (even private code). 62
  • 208. What the tutorial teaches myproject/ settings.py urls.py myapp/ models.py mysecondapp/ views.py … 63
  • 209. from myproject.myapp.models import … from myproject. myapp.models import … … myproject.settings myproject.urls 64
  • 210. Project coupling kills re-use 65
  • 211. Projects in real life. 66
  • 212. Projects in real life. • A settings module. 66
  • 213. Projects in real life. • A settings module. • A root URLConf. 66
  • 214. Projects in real life. • A settings module. • A root URLConf. • Maybe a manage.py (but…) 66
  • 215. Projects in real life. • A settings module. • A root URLConf. • Maybe a manage.py (but…) • And that’s it. 66
  • 216. Advantages 67
  • 217. Advantages • No assumptions about where things live. 67
  • 218. Advantages • No assumptions about where things live. • No PYTHONPATH magic. 67
  • 219. Advantages • No assumptions about where things live. • No PYTHONPATH magic. • Reminds you that “projects” are just a Python module. 67
  • 220. You don’t even need a project 68
  • 225. Where apps really live 70
  • 226. Where apps really live • Single module directly on Python path (registration, tagging, etc.). 70
  • 227. Where apps really live • Single module directly on Python path (registration, tagging, etc.). • Related modules under a top-level package (ellington.events, ellington.podcasts, etc.) 70
  • 228. Where apps really live • Single module directly on Python path (registration, tagging, etc.). • Related modules under a top-level package (ellington.events, ellington.podcasts, etc.) • No projects (ellington.settings doesn’t exist). 70
  • 230. Want to distribute? • Build a package with distutils/setuptools. 71
  • 231. Want to distribute? • Build a package with distutils/setuptools. • Put it on PyPI (or a private package server). 71
  • 232. Want to distribute? • Build a package with distutils/setuptools. • Put it on PyPI (or a private package server). • Now it works with easy_install, pip, buildout, … 71
  • 234. General best practices • Establish dependancy rules. 72
  • 235. General best practices • Establish dependancy rules. • Establish a minimum Python version (suggestion: Python 2.5). 72
  • 236. General best practices • Establish dependancy rules. • Establish a minimum Python version (suggestion: Python 2.5). • Establish a minimum Django version (suggestion: Django 1.0). 72
  • 237. General best practices • Establish dependancy rules. • Establish a minimum Python version (suggestion: Python 2.5). • Establish a minimum Django version (suggestion: Django 1.0). • Test frequently against new versions of dependancies. 72
  • 241. Don’t touch! • Good applications are extensible without patching. 75
  • 242. Don’t touch! • Good applications are extensible without patching. • Take advantage of every extensibility point an application gives you. 75
  • 243. Don’t touch! • Good applications are extensible without patching. • Take advantage of every extensibility point an application gives you. • You may end up doing something that deserves a new application anyway. 75
  • 244. But this application wasn’t meant to be extended! 76
  • 247. Extending a view • Wrap the view with your own code. 78
  • 248. Extending a view • Wrap the view with your own code. • Doing it repetitively? Write a decorator. 78
  • 250. Extending a model • Relate other models to it. 79
  • 251. Extending a model • Relate other models to it. • Subclass it. 79
  • 252. Extending a model • Relate other models to it. • Subclass it. • Proxy subclasses (Django 1.1). 79
  • 254. Extending a form • Subclass it. 80
  • 255. Extending a form • Subclass it. • There is no step 2. 80
  • 257. Other tricks • Signals lets you fire off customized behavior when certain events happen. 81
  • 258. Other tricks • Signals lets you fire off customized behavior when certain events happen. • Middleware offers full control over request/response handling. 81
  • 259. Other tricks • Signals lets you fire off customized behavior when certain events happen. • Middleware offers full control over request/response handling. • Context processors can make additional information available if a view doesn’t. 81
  • 260. If you must make changes to external code… 82
  • 261. Keep changes to a minimum 83
  • 262. Keep changes to a minimum • If possible, instead of adding a feature, add extensibility. 83
  • 263. Keep changes to a minimum • If possible, instead of adding a feature, add extensibility. • Keep as much changed code as you can out of the original app. 83
  • 265. Stay up-to-date • Don’t want to get out of sync with the original version of the code! 84
  • 266. Stay up-to-date • Don’t want to get out of sync with the original version of the code! • You might miss bugfixes. 84
  • 267. Stay up-to-date • Don’t want to get out of sync with the original version of the code! • You might miss bugfixes. • You might even miss the feature you needed. 84
  • 268. Use a good VCS 85
  • 269. Use a good VCS • Subversion vendor branches don’t cut it. 85
  • 270. Use a good VCS • Subversion vendor branches don’t cut it. • DVCSes are perfect for this: 85
  • 271. Use a good VCS • Subversion vendor branches don’t cut it. • DVCSes are perfect for this: • Mercurial queues. 85
  • 272. Use a good VCS • Subversion vendor branches don’t cut it. • DVCSes are perfect for this: • Mercurial queues. • Git rebasing. 85
  • 273. Use a good VCS • Subversion vendor branches don’t cut it. • DVCSes are perfect for this: • Mercurial queues. • Git rebasing. • At the very least, maintain a patch queue by hand. 85
  • 274. Be a good citizen 86
  • 275. Be a good citizen • If you change someone else’s code, let them know. 86
  • 276. Be a good citizen • If you change someone else’s code, let them know. • Maybe they’ll merge your changes in and you won’t have to fork anymore. 86
  • 278. Deployment 88
  • 280. Deployment should... • Be automated. 89
  • 281. Deployment should... • Be automated. • Automatically manage dependencies. 89
  • 282. Deployment should... • Be automated. • Automatically manage dependencies. • Be isolated. 89
  • 283. Deployment should... • Be automated. • Automatically manage dependencies. • Be isolated. • Be repeatable. 89
  • 284. Deployment should... • Be automated. • Automatically manage dependencies. • Be isolated. • Be repeatable. • Be identical in staging and in production. 89
  • 285. Deployment should... • Be automated. • Automatically manage dependencies. • Be isolated. • Be repeatable. • Be identical in staging and in production. • Work the same for everyone. 89
  • 286. Dependency Isolation Automation management 90
  • 287. Dependency Isolation Automation management 90
  • 288. Dependency Isolation Automation management apt/yum/... virtualenv Capistrano easy_install zc.buildout Fabric pip Puppet/Chef/… zc.buildout 90
  • 289. Dependency Isolation Automation management apt/yum/... virtualenv Capistrano easy_install zc.buildout Fabric pip Puppet/Chef/… zc.buildout 90
  • 290. Dependency Isolation Automation management apt/yum/... virtualenv Capistrano easy_install zc.buildout Fabric pip Puppet/Chef/… zc.buildout 90
  • 292. Dependancy management • The Python ecosystem rocks! 91
  • 293. Dependancy management • The Python ecosystem rocks! • Python package management doesn’t. 91
  • 294. Dependancy management • The Python ecosystem rocks! • Python package management doesn’t. • Installing packages — and dependancies — correctly is a lot harder than it should be; most defaults are wrong. 91
  • 295. Dependancy management • The Python ecosystem rocks! • Python package management doesn’t. • Installing packages — and dependancies — correctly is a lot harder than it should be; most defaults are wrong. • Here be dragons. 91
  • 297. Vendor packages • APT, Yum, … 92
  • 298. Vendor packages • APT, Yum, … • The good: familiar tools; stability; handles dependancies not on PyPI. 92
  • 299. Vendor packages • APT, Yum, … • The good: familiar tools; stability; handles dependancies not on PyPI. • The bad: small selection; not (very) portable; hard to supply user packages. 92
  • 300. Vendor packages • APT, Yum, … • The good: familiar tools; stability; handles dependancies not on PyPI. • The bad: small selection; not (very) portable; hard to supply user packages. • The ugly: installs packages system-wide. 92
  • 302. easy_install • The good: multi-version packages. 93
  • 303. easy_install • The good: multi-version packages. • The bad: requires ‘net connection; can’t uninstall; can’t handle non-PyPI packages; multi-version packages barely work. 93
  • 304. easy_install • The good: multi-version packages. • The bad: requires ‘net connection; can’t uninstall; can’t handle non-PyPI packages; multi-version packages barely work. • The ugly: stale; unsupported; defaults almost totally wrong; installs system-wide. 93
  • 306. pip http://pip.openplans.org/ • “Pip Installs Packages” 94
  • 307. pip http://pip.openplans.org/ • “Pip Installs Packages” • The good: Just Works™; handles non- PyPI packages (including direct from SCM); repeatable dependancies; integrates with virtualenv for isolation. 94
  • 308. pip http://pip.openplans.org/ • “Pip Installs Packages” • The good: Just Works™; handles non- PyPI packages (including direct from SCM); repeatable dependancies; integrates with virtualenv for isolation. • The bad: still young; not yet bundled. 94
  • 309. pip http://pip.openplans.org/ • “Pip Installs Packages” • The good: Just Works™; handles non- PyPI packages (including direct from SCM); repeatable dependancies; integrates with virtualenv for isolation. • The bad: still young; not yet bundled. • The ugly: haven’t found it yet. 94
  • 311. zc.buildout http://buildout.org/ • The good: incredibly flexible; handles any sort of dependancy; repeatable builds; reusable “recipes;” good ecosystem; handles isolation, too. 95
  • 312. zc.buildout http://buildout.org/ • The good: incredibly flexible; handles any sort of dependancy; repeatable builds; reusable “recipes;” good ecosystem; handles isolation, too. • The bad: often cryptic, INI-style configuration file; confusing duplication of recipes; sometimes too flexible. 95
  • 313. zc.buildout http://buildout.org/ • The good: incredibly flexible; handles any sort of dependancy; repeatable builds; reusable “recipes;” good ecosystem; handles isolation, too. • The bad: often cryptic, INI-style configuration file; confusing duplication of recipes; sometimes too flexible. • The ugly: nearly completely undocumented. 95
  • 316. Package isolation • Why? • Site A requires Foo v1.0; site B requires Foo v2.0. 96
  • 317. Package isolation • Why? • Site A requires Foo v1.0; site B requires Foo v2.0. • You need to develop against multiple versions of dependancies. 96
  • 319. Package isolation tools • Virtual machines (Xen, VMWare, EC2, …) 97
  • 320. Package isolation tools • Virtual machines (Xen, VMWare, EC2, …) • Multiple Python installations. 97
  • 321. Package isolation tools • Virtual machines (Xen, VMWare, EC2, …) • Multiple Python installations. • “Virtual” Python installations. 97
  • 322. Package isolation tools • Virtual machines (Xen, VMWare, EC2, …) • Multiple Python installations. • “Virtual” Python installations. • virtualenv http://pypi.python.org/pypi/virtualenv 97
  • 323. Package isolation tools • Virtual machines (Xen, VMWare, EC2, …) • Multiple Python installations. • “Virtual” Python installations. • virtualenv http://pypi.python.org/pypi/virtualenv • zc.buildout http://buildout.org/ 97
  • 325. Why automate? • “I can’t push this fix to the servers until Alex gets back from lunch.” 98
  • 326. Why automate? • “I can’t push this fix to the servers until Alex gets back from lunch.” • “Sorry, I can’t fix that. I’m new here.” 98
  • 327. Why automate? • “I can’t push this fix to the servers until Alex gets back from lunch.” • “Sorry, I can’t fix that. I’m new here.” • “Oops, I just made the wrong version of our site live.” 98
  • 328. Why automate? • “I can’t push this fix to the servers until Alex gets back from lunch.” • “Sorry, I can’t fix that. I’m new here.” • “Oops, I just made the wrong version of our site live.” • “It’s broken! What’d you do!?” 98
  • 330. Automation basics • SSH is right out. 99
  • 331. Automation basics • SSH is right out. • Don’t futz with the server. Write a recipe. 99
  • 332. Automation basics • SSH is right out. • Don’t futz with the server. Write a recipe. • Deploys should be idempotent. 99
  • 334. Capistrano http://capify.org/ • The good: lots of features; good documentation; active community. 100
  • 335. Capistrano http://capify.org/ • The good: lots of features; good documentation; active community. • The bad: stale development; very “opinionated” and Rails-oriented. 100
  • 337. Fabric http://fabfile.org/ • The good: very simple; flexible; actively developed; Python. 101
  • 338. Fabric http://fabfile.org/ • The good: very simple; flexible; actively developed; Python. • The bad: no high-level commands; in flux. 101
  • 340. Configuration management • CFEngine, Puppet, Chef, … 102
  • 341. Configuration management • CFEngine, Puppet, Chef, … • Will handle a lot more than code deployment! 102
  • 342. Configuration management • CFEngine, Puppet, Chef, … • Will handle a lot more than code deployment! • I only know a little about these. 102
  • 343. Recommendations Pip, Virtualenv, and Fabric Buildout and Fabric. Buildout and Puppet/Chef/…. Utility computing and Puppet/Chef/…. 103
  • 345. net. LiveJournal Backend: Today (Roughly.) BIG-IP perlbal (httpd/proxy) Global Database bigip1 mod_perl bigip2 proxy1 master_a master_b web1 proxy2 web2 proxy3 web3 Memcached slave1 slave2 ... slave5 djabberd proxy4 mc1 web4 djabberd proxy5 ... mc2 User DB Cluster 1 djabberd webN mc3 uc1a uc1b mc4 User DB Cluster 2 ... uc2a uc2b gearmand Mogile Storage Nodes gearmand1 mcN User DB Cluster 3 sto1 sto2 gearmandN uc3a uc3b Mogile Trackers ... sto8 tracker1 tracker3 User DB Cluster N ucNa ucNb MogileFS Database “workers” gearwrkN Job Queues (xN) mog_a mog_b theschwkN jqNa jqNb slave1 slaveN http://danga.com/words/ Brad Fitzpatrik, http://danga.com/words/2007_06_usenix/ 3 105
  • 348. Application servers • Apache + mod_python 107
  • 349. Application servers • Apache + mod_python • Apache + mod_wsgi 107
  • 350. Application servers • Apache + mod_python • Apache + mod_wsgi • Apache/lighttpd + FastCGI 107
  • 351. Application servers • Apache + mod_python • Apache + mod_wsgi • Apache/lighttpd + FastCGI • SCGI, AJP, nginx/mod_wsgi, ... 107
  • 352. Use mod_wsgi 108
  • 354. import os, sys # Add to PYTHONPATH whatever you need sys.path.append('/usr/local/django') # Set DJANGO_SETTINGS_MODULE os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings' # Create the application for mod_wsgi import django.core.handlers.wsgi application = django.core.handlers.wsgi.WSGIHandler() 110
  • 355. “Scale” 111
  • 356. Does this scale? django database media server 112
  • 357. Does this scale? django database media server Maybe! 112
  • 358. Things per secong Number of things 113
  • 359. Real-world example Database A 175 req/s Database B 75 req/s 114
  • 360. Real-world example http://tweakers.net/reviews/657/6 115
  • 361. django media web server database database server 116
  • 363. Why separate hardware? • Resource contention 117
  • 364. Why separate hardware? • Resource contention • Separate performance concerns 117
  • 365. Why separate hardware? • Resource contention • Separate performance concerns • 0 → 1 is much harder than 1 → N 117
  • 369. Connection middleware • Proxy between web and database layers 119
  • 370. Connection middleware • Proxy between web and database layers • Most implement hot fallover and connection pooling 119
  • 371. Connection middleware • Proxy between web and database layers • Most implement hot fallover and connection pooling • Some also provide replication, load balancing, parallel queries, connection limiting, &c 119
  • 372. Connection middleware • Proxy between web and database layers • Most implement hot fallover and connection pooling • Some also provide replication, load balancing, parallel queries, connection limiting, &c • DATABASE_HOST = '127.0.0.1' 119
  • 375. Connection middleware • PostgreSQL: pgpool • MySQL: MySQL Proxy 120
  • 376. Connection middleware • PostgreSQL: pgpool • MySQL: MySQL Proxy • Database-agnostic: sqlrelay 120
  • 377. Connection middleware • PostgreSQL: pgpool • MySQL: MySQL Proxy • Database-agnostic: sqlrelay • Oracle: ? 120
  • 378. django media web server media server database database server 121
  • 381. Media server traits • Fast • Lightweight 122
  • 382. Media server traits • Fast • Lightweight • Optimized for high concurrency 122
  • 383. Media server traits • Fast • Lightweight • Optimized for high concurrency • Low memory overhead 122
  • 384. Media server traits • Fast • Lightweight • Optimized for high concurrency • Low memory overhead • Good HTTP citizen 122
  • 388. Media servers • Apache? • lighttpd • nginx 123
  • 389. Media servers • Apache? • lighttpd • nginx • S3 123
  • 390. The absolute minimum django media web server media server database database server 124
  • 391. The absolute minimum django media database web server 125
  • 392. proxy media load balancer media server django django django web server cluster database database server 126
  • 395. Load balancer traits • Low memory overhead 128
  • 396. Load balancer traits • Low memory overhead • High concurrency 128
  • 397. Load balancer traits • Low memory overhead • High concurrency • Hot fallover 128
  • 398. Load balancer traits • Low memory overhead • High concurrency • Hot fallover • Other nifty features... 128
  • 400. Load balancers • Apache + mod_proxy 129
  • 401. Load balancers • Apache + mod_proxy • perlbal 129
  • 402. Load balancers • Apache + mod_proxy • perlbal • nginx 129
  • 403. Load balancers • Apache + mod_proxy • perlbal • nginx • Varnish 129
  • 404. Load balancers • Apache + mod_proxy • perlbal • nginx • Varnish • Squid 129
  • 405. CREATE POOL mypool POOL mypool ADD 10.0.0.100 POOL mypool ADD 10.0.0.101 CREATE SERVICE mysite SET listen = my.public.ip SET role = reverse_proxy SET pool = mypool SET verify_backend = on SET buffer_size = 120k ENABLE mysite 130
  • 406. you@yourserver:~$ telnet localhost 60000 pool mysite add 10.0.0.102 OK nodes 10.0.0.101 10.0.0.101 lastresponse 1237987449 10.0.0.101 requests 97554563 10.0.0.101 connects 129242435 10.0.0.101 lastconnect 1237987449 10.0.0.101 attempts 129244743 10.0.0.101 responsecodes 200 358 10.0.0.101 responsecodes 302 14 10.0.0.101 responsecodes 207 99 10.0.0.101 responsecodes 301 11 10.0.0.101 responsecodes 404 18 10.0.0.101 lastattempt 1237987449 131
  • 407. proxy proxy proxy media media load balancing cluster media server cluster django django django cache cache web server cluster cache cluster database database database database server cluster 132
  • 409. BALANCE = None def balance_sheet(request): global BALANCE if not BALANCE: bank = Bank.objects.get(...) BALANCE = bank.total_balance() ... 134
  • 410. BALANCE = None def balance_sheet(request): global BALANCE if not BALANCE: bank = Bank.objects.get(...) BALANCE = bank.total_balance() ... FAIL 134
  • 411. Global variables are right out 135
  • 412. from django.cache import cache def balance_sheet(request): balance = cache.get('bank_balance') if not balance: bank = Bank.objects.get(...) balance = bank.total_balance() cache.set('bank_balance', balance) ... 136
  • 413. from django.cache import cache def balance_sheet(request): balance = cache.get('bank_balance') if not balance: bank = Bank.objects.get(...) balance = bank.total_balance() cache.set('bank_balance', balance) ... WIN 136
  • 414. def generate_report(request): report = get_the_report() open('/tmp/report.txt', 'w').write(report) return redirect(view_report) def view_report(request): report = open('/tmp/report.txt').read() return HttpResponse(report) 137
  • 415. def generate_report(request): report = get_the_report() open('/tmp/report.txt', 'w').write(report) return redirect(view_report) def view_report(request): report = open('/tmp/report.txt').read() return HttpResponse(report) FAIL 137
  • 418. Further reading • Cal Henderson, Building Scalable Web Sites 139
  • 419. Further reading • Cal Henderson, Building Scalable Web Sites • John Allspaw, The Art of Capacity Planning 139
  • 420. Further reading • Cal Henderson, Building Scalable Web Sites • John Allspaw, The Art of Capacity Planning • http://kitchensoap.com/ 139
  • 421. Further reading • Cal Henderson, Building Scalable Web Sites • John Allspaw, The Art of Capacity Planning • http://kitchensoap.com/ • http://highscalability.com/ 139
  • 422. Monitoring 140
  • 423. Goals 141
  • 424. Goals • When the site goes down, know it immediately. 141
  • 425. Goals • When the site goes down, know it immediately. • Automatically handle common sources of downtime. 141

Notes de l'éditeur

  1. Oh hai. I’m Jacob Kaplan-Moss; here’s a few ways to get in touch with me. I wear quite a few hats. Today, I’m wearing two: I’m one of the lead developers of Django; I’m a partner at `Revolution Systems`__, a consulting business specializing in, well, the stuff I’m about to talk about.
  2. Just a brief shameless plug: my day job is as a consultant working in and around these issues. I get to come talk about this stuff publicly because of this gig, so If you need info specific to your situation please consider giving us a call!
  3. Credit where credit is due: I first gave this talk at PyCon 2009 with the help of `James Bennett`__, so much of my thinking and some of the content comes from James. __ http://b-list.org/
  4. This is the main theme of this tutorial: you’ve written a Django site...
  5. ... and you’ve barely started. Writing the app is the easy part; what comes next is seriously complicated.
  6. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  7. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  8. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  9. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  10. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  11. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  12. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  13. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  14. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  15. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  16. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  17. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  18. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  19. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  20. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  21. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  22. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  23. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  24. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  25. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  26. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  27. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  28. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  29. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  30. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  31. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  32. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  33. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  34. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  35. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  36. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  37. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  38. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  39. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  40. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  41. Here’s just one list (Leonard Lin’s) of what you might need to think about *after* writing your app; I could easily add another dozen more things to this list. Luckily, not every site needs *all* of this... but any site of any size will need to understand at last some of this stuff, and bigger deployments will need to worry about all of it. Obviously this is far too much to cover even in a long tutorial, so we’ll pick and choose; the point is that writing the app is usually the easy part. Today we’re talking about what comes next.
  42. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  43. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  44. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  45. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  46. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  47. So here’s the bare minimum you’ll need to think about: * You need to test your sites. Without tests, you can’t move forward without downtime. * You need to write your code in a way that makes deployment possible — and easy. * You need to use deployment tools that automate the repetitive bits and keep you from shooting yourself in the foot. * You need a production environment that scales with your traffic. * You need to watch your sites in action so that you know what’s up. * When things go slow you need to fix it.
  48. So now you’ve got a working app, but how do you *know* that it works? Sure, you could just run the site and click around, but that guarantees nothing. The difference between stable and unstable code is often the quality of the tests. The “fear” that Beck is talking about here is the fear of not knowing if your new site will run or not. Testing, while often boring, relives you of that fear.
  49. `Test Driven Development`__ by Kent Beck is the bible of TDD. Beck requires that you only add new code to your project after an existing test fails. This means writing tests before writing any new code. It can be rewarding -- particularly if you’re pair programming so you can keep each other honest. Unfortunately, it’s also simply not practical for most developers. __ http://www.amazon.com/dp/0321146530/
  50. This is the pragmatic approach to testing: write tests when you need them. If you need to chase down a bug, write a test to trigger the bug, then when the test passes you know the bug is licked. Yes, this leaves you with less than perfect coverage.
  51. I’m speaking from experience here. Once you commit with even a *single* breaking test, finding the time to clean up is an uphill battle. Knowing the suite fails discourages you from even running the tests at all, and thus the quality continues to decline.
  52. I tend to think of this as a continuum of testing; another way of thinking about this is a continuum between internal/external, or whitebox/blackbox.
  53. I tend to think of this as a continuum of testing; another way of thinking about this is a continuum between internal/external, or whitebox/blackbox.
  54. I tend to think of this as a continuum of testing; another way of thinking about this is a continuum between internal/external, or whitebox/blackbox.
  55. I tend to think of this as a continuum of testing; another way of thinking about this is a continuum between internal/external, or whitebox/blackbox.
  56. Testing web applications is *hard*. Anything with a persistent state -- i.e. a database -- and network communications -- HTTP -- is a real pain to test. Ruby on Rails really changed the game, though. When it shipped a set of tools that took most of the pain out of automating web tests, it galvenized developers to compete. Django’s since come out with our own spin on this set of tools. I’m only going to cover doctests in this intro tutorial, so a brief overview of the other tools is in order: * Unit tests: Django can run tests using Python’s standard `unittest` framework, which is based on Java’s JUnit. Perfect for more formal unit tests like you’d see in Java/C#. * Fixtures: fixtures are a way of loading preset data into the database for each test, thus letting you test against “real” data. * The test client: this lets you “fake” a request to your views and inspect the returned response, template, and context. * Email capture: the test suite will intercept sent email so you can test logic involving email.
  57. Testing web applications is *hard*. Anything with a persistent state -- i.e. a database -- and network communications -- HTTP -- is a real pain to test. Ruby on Rails really changed the game, though. When it shipped a set of tools that took most of the pain out of automating web tests, it galvenized developers to compete. Django’s since come out with our own spin on this set of tools. I’m only going to cover doctests in this intro tutorial, so a brief overview of the other tools is in order: * Unit tests: Django can run tests using Python’s standard `unittest` framework, which is based on Java’s JUnit. Perfect for more formal unit tests like you’d see in Java/C#. * Fixtures: fixtures are a way of loading preset data into the database for each test, thus letting you test against “real” data. * The test client: this lets you “fake” a request to your views and inspect the returned response, template, and context. * Email capture: the test suite will intercept sent email so you can test logic involving email.
  58. Testing web applications is *hard*. Anything with a persistent state -- i.e. a database -- and network communications -- HTTP -- is a real pain to test. Ruby on Rails really changed the game, though. When it shipped a set of tools that took most of the pain out of automating web tests, it galvenized developers to compete. Django’s since come out with our own spin on this set of tools. I’m only going to cover doctests in this intro tutorial, so a brief overview of the other tools is in order: * Unit tests: Django can run tests using Python’s standard `unittest` framework, which is based on Java’s JUnit. Perfect for more formal unit tests like you’d see in Java/C#. * Fixtures: fixtures are a way of loading preset data into the database for each test, thus letting you test against “real” data. * The test client: this lets you “fake” a request to your views and inspect the returned response, template, and context. * Email capture: the test suite will intercept sent email so you can test logic involving email.
  59. Testing web applications is *hard*. Anything with a persistent state -- i.e. a database -- and network communications -- HTTP -- is a real pain to test. Ruby on Rails really changed the game, though. When it shipped a set of tools that took most of the pain out of automating web tests, it galvenized developers to compete. Django’s since come out with our own spin on this set of tools. I’m only going to cover doctests in this intro tutorial, so a brief overview of the other tools is in order: * Unit tests: Django can run tests using Python’s standard `unittest` framework, which is based on Java’s JUnit. Perfect for more formal unit tests like you’d see in Java/C#. * Fixtures: fixtures are a way of loading preset data into the database for each test, thus letting you test against “real” data. * The test client: this lets you “fake” a request to your views and inspect the returned response, template, and context. * Email capture: the test suite will intercept sent email so you can test logic involving email.
  60. Testing web applications is *hard*. Anything with a persistent state -- i.e. a database -- and network communications -- HTTP -- is a real pain to test. Ruby on Rails really changed the game, though. When it shipped a set of tools that took most of the pain out of automating web tests, it galvenized developers to compete. Django’s since come out with our own spin on this set of tools. I’m only going to cover doctests in this intro tutorial, so a brief overview of the other tools is in order: * Unit tests: Django can run tests using Python’s standard `unittest` framework, which is based on Java’s JUnit. Perfect for more formal unit tests like you’d see in Java/C#. * Fixtures: fixtures are a way of loading preset data into the database for each test, thus letting you test against “real” data. * The test client: this lets you “fake” a request to your views and inspect the returned response, template, and context. * Email capture: the test suite will intercept sent email so you can test logic involving email.