Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Scalable web application architecture
Similaire à Scalable web application architecture (20)
Dernier
Dernier (20)
Scalable web application architecture
- 1. SCALABLE WEB APPLICATION ARCHITECTURE (USING DJANGO + EXTJS) Michał Karzyński, PyWaw #27
- 2. TALK OUTLINE 1. Application architecture 2. Backend code organization (Django) 3. Frontend code organization (ExtJS) 4. Communication APIs 5. Deployment and Scaling
- 3. YOURSTRULY Michał Karzyński • project manager at Politechnika Gdańska • freelance developer and consultant • polyglot, currently: Python and JavaScript • web developer since 1996 • @postrational • http://michal.karzynski.pl
- 4. APPLICATION ARCHITECTURE Backend API Frontend Server Cloud Browser Python JSON JavaScript Django ExtJS I/O GUITransport
- 5. BACKEND (DJANGO) CODE STRUCTURE Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware HTTP Request HTTP Response Model (ORM)
- 6. MIDDLEWARE Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM) • Processes incoming requests and outgoing responses • Hooks in before or after the controller function • e.g. SessionMiddleware, CacheMiddleware
- 7. MIDDLEWARE EXAMPLE class SessionMiddleware(object): def process_request(self, request): engine = import_module(settings.SESSION_ENGINE) session_key = request.COOKIES.get( settings.SESSION_COOKIE_NAME, None) request.session = engine.SessionStore(session_key)
- 8. ROUTER • Maps URL patterns to controller function calls • Uses regular expressions • Can use positional and named parameters • Routes with default parameter values Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM)
- 9. ROUTER EXAMPLE urlpatterns = patterns('myapp.views', url(r'^$', home_page), # simple pattern url(r'^admin/', include(admin.site.urls)), # pattern inclusion url(r'^item/(w+)/(d+)/$', display_item), # positional params url(r'^info/(?P<slug>[-w]+)/$', display_page), # named params url(r'^info/$', display_page, { 'slug' : 'index' }), # default value )
- 10. DECORATORS • Wrap controller functions in additional logic • Can modify request object or cause a redirect • e.g. login_required, permission_required, cache_page Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM)
- 11. CONTROLLER (VIEW) • Main application logic • Functions receive an HTTP request and returns an HTTP response • Communicates with the model layer, session store, etc. Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM)
- 12. CONTROLLER (VIEW) EXAMPLE def display_institute(request, id): institute = get_object_or_404(Institute, pk=safeint(id)) return render_to_response('institute_display.html', { 'institute' : institute })
- 13. CONTROLLER (VIEW) EXAMPLE def save_offer(self, request): form_data = json_deserialize(request.read()) # Validate incoming offer data offer_form = OfferForm(form_data) validation_errors = offer_form.errors if not validation_errors: offer = offer_form.save() response = { 'success' : True, 'offer_url' : offer.get_absolute_url() } if validation_errors: for key in validation_errors.keys(): validation_errors[key] = " ".join(validation_errors[key]) response = { 'success' : False, 'errors' : validation_errors } return HttpResponse(json_serialize(response), mimetype='application/json')
- 14. MODEL (ORM) • Database abstraction layer • Converts DB rows into Python objects • Generates DB structure and lazy-loading SQL queries Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM)
- 15. MODEL (ORM) EXAMPLE class Person(models.Model): first_name = models.CharField(_('first name'), max_length=30, blank=True) last_name = models.CharField(_('last name'), max_length=30, blank=True) email = models.EmailField(_('email address'), blank=True) def get_full_name(self): """ Returns the first_name plus the last_name, with a space in between. """ full_name = '%s %s' % (self.first_name, self.last_name) return full_name.strip() Offer.objects.filter(deadline__date__gte=datetime.date.today()) .distinct().annotate(closest_deadline=Min('deadline__date')) .order_by('closest_deadline').select_related()
- 16. TEMPLATE • Generate HTML • Use Django’s templating language Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM)
- 17. TEMPLATE EXAMPLE {% extends "base.html" %} {% load markup_textile %} {% block title %}{{ page.title }} {% endblock %} {% block body %} <div class="page-header"> <h3>{{ page.title }}</h3> </div> <div class="span7"> <p> {{ page.body|textile }} </p> </div> {% if page.comments %} <div id="disqus_thread" class="span7"></div> {% endif %} {% endblock %}
- 18. SERIALIZER • Convert objects into strings for easy transport and data exchange Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model (ORM) return HttpResponse(json_serialize(data), mimetype='application/json')
- 19. APPLICATION ARCHITECTURE Django ExtJS Decorators Router Middleware Controller (a.k.aView) Template / Serializer Middleware Model Controller View Store Browser event click API request response view update Model (ORM)
- 21. VIEW Model Controller View Store HTTP request HTTP response event view update • Sets up visible GUI widgets • Binds collection views to data stores
- 22. VIEW EXAMPLE Ext.create('Ext.grid.Panel', { title: 'People', alias : 'widget.peoplepanel', store: Ext.data.StoreManager.lookup('peopleStore'), columns: [ { text: 'First name', dataIndex: 'first_name' }, { text: 'Last name', dataIndex: 'last_name' }, { text: 'Email', dataIndex: 'email', flex: 1 } ], height: 200, width: 400, renderTo: Ext.getBody() });
- 23. VIEW EXAMPLE
- 24. CONTROLLER • Main application logic • Sets up event listeners • Interacts with models • ...or sends custom requests to the API, processes responses, updates the GUI Model Controller View Store HTTP request HTTP response event view update
- 25. CONTROLLER EXAMPLE Ext.define('MyApp.controller.People', { extend:'Ext.app.Controller', init: function() { this.control({ 'peoplepanel':{ itemclick : this.onPersonClicked } }); }, onPersonClicked: function(view, record, item, index, e, eOpts){ console.log('Person item clicked:' record.get('firstName')); record.set('was_clicked', true); record.save(); } });
- 26. MODEL • Represents API data as JavaScript objects • Client-side representation of server-side models • Performs data validation Model Controller View Store HTTP request HTTP response event view update
- 27. MODEL EXAMPLE Ext.define('Person', { extend: 'Ext.data.Model', idProperty: 'id', fields: [ {name: 'id', type: 'int'}, {name: 'first_name', type: 'string'}, {name: 'last_name', type: 'string'}, {name: 'age', type: 'int'}, {name: 'was_clicked',type: 'bool'} ], get_full_name: function() { return this.get('first_name') + ' ' + this.get('last_name'); } });
- 28. STORE • Communicates with the API • Enables UI data binding • Converts API responses to instances of models Model Controller View Store HTTP request HTTP response event view update
- 29. STORE EXAMPLE Ext.define('MyApp.store.Persons',{ extend : 'Ext.data.Store', storeId:'peopleStore', model: 'MyApp.model.Person' proxy: { type: 'ajax', api: { create : '/api/person/new', read : '/api/person/load', update : '/api/person/update', destroy : '/api/person/destroy' }, reader: { type: 'json', root: 'person', successProperty: 'success' } } });
- 30. HTTP API DESIGN headers body method URL headers body status HTTP request HTTP response Cookies User-agent Language prefs GET, POST, PUT, DELETE, etc. protocol, host port, resource, GET params uploaded files, JSON data, etc. Cookies Cache-control 200, 302, 404, 500, etc. HTML, JSON, binary data. etc.
- 32. AJAXIFIED WEBSITE request response headers POST parameters POST URL headers partial HTML 200
- 33. REST request response Resource GET PUT POST DELETE Collection URI List elements Replace collection Create element Delete collection Element URI Retrieve element Replace or create Delete element headers object JSON method resource headers object JSON 200, 201, 400, 409, etc.
- 34. SOAP / JSON-RPC REMOTE PROCEDURE CALL request response headers method-call and parameters in an XML or JSON envelope POST API endpoint headers method response object in an XML or JSON envelope 200
- 35. EXTJS-STYLE API request response headers function arguments JSON method function URI headers response object JSON 200
- 36. Demo
- 38. SCALING Step 1: Optimize your code • optimize number of DB queries • rewrite slower ORM queries as native SQL (stored procedures) • use cacheing Gunicorn Nginx Django PostgreSQL Virtualenv
- 39. SCALING Step 2:Add Redis • in memory key-value store • as session store • as cache backend Gunicorn Nginx Django Redis PostgreSQL Virtualenv
- 40. SCALING Step 3:Add Varnish • in-memory cache • speed up serving static files Gunicorn Nginx Django Redis PostgreSQL Virtualenv Varnish
- 41. SCALING Step 4: Split up your servers Gunicorn Nginx Django Virtualenv Varnish Nginx Static files Varnish PostgreSQLRedis Backend Frontend DBCache
- 42. SCALING Step 5: Clusters PostgreSQL Redis App-node cluster DB ClusterRedis cluster (coming soon) Gunicorn Nginx Django Virtualenv Varnish Gunicorn Nginx Django Virtualenv Nginx Static files Varnish Nginx Static files Static files cluster or use a CDN Redis PostgreSQL