SlideShare une entreprise Scribd logo
1  sur  24
State of WSGI 2
    Graham Dumpleton
   US PyCon March 2012

      @GrahamDumpleton
  Graham.Dumpleton@gmail.com
1.0 (PEP 333)


PEP created December 2003.

A series of compromises.

  Iterable vs write().

  Limited async capability.
1.0.1 (PEP 3333)

PEP created September 2010.

Clarifications to PEP 333 to handle str/bytes
distinction in Python 3.X.

Apache/mod_wsgi had already been
providing support for Python 3.X since April
2009.

Was a quick and painless process. NOT!
Shortcomings



Hard to write middleware correctly.
Easy targets

Drop start_response() and write().
def app(environ):
    status = '200 OK'
    content = 'Hello world'

    headers = [
        ('Content-type', 'text/plain'),
        ('Content-length', str(len(content)))]

    return (status, headers, [content])
Unwanted advances

Support for async systems.

  The killer for async is that how you
  interact with wsgi.input means it is
  blocking.

  Hard to change it to make it work for
  async systems without making it too
  complicated for sync systems.
Wishful thinking


Standardised high level request/response
objects.

  WebOb

  Werkzeug
Interesting problems
Cleaning up at end of request.

Middleware and response content length.

Unknown request content length.

  No compressed request content.

  No chunked requests.

  No full duplex HTTP.
Has the boat sailed?

Too much legacy code relying on WSGI 1.0.

Potential missed opportunity for significant
change when updating specification for
Python 3.X.

The possibilities of what could have been
done are now probably only of academic
interest.
Ideas I toyed with

Use context managers to improve resource
management.

Implementing 'wsgi.input' as an iterable.

Support for chunked request content.

Iterating through a list of applications until
one agrees to handle it.
Resource management


Need to override close() of the iterable.

Way too complicated for mere mortals.

Makes wsgi.file_wrapper impotent.
How complicated

class Generator:
    def __init__(self, iterable, callback, environ):
        self.__iterable = iterable
        self.__callback = callback
        self.__environ = environ
    def __iter__(self):
        for item in self.__iterable:
             yield item
    def close(self):
        try:
             if hasattr(self.__iterable, 'close'):
                 self.__iterable.close()
        finally:
             self.__callback(self.__environ)
Not done yet

class ExecuteOnCompletion:
    def __init__(self, application, callback):
        self.__application = application
        self.__callback = callback
    def __call__(self, environ, start_response):
        try:
             result = self.__application(environ, start_response)
        except:
             self.__callback(environ)
             raise
        return Generator(result, self.__callback, environ)
Response as object

Make response a context manager.

Response content is an attribute.

Consumer of response executes:
  with response:
      for data in response.content:
          process(data)


Can override __enter__()/__exit__().
Request as object
Also make request a context manager.

Request content is an attribute, but also
make it an iterable.

Reading all request content becomes:
  with request:
     content = ''.join(request.content)


Content length becomes advisory only, so can
implement chunked content and mutating
input filters.
Hello world

def application(request, response):

    status = '200 OK'
    headers = [('Content-type', 'text/plain'),]
    output = 'Hello world!'

    return response(status, headers, [output])
Consuming input
def application(request, response):

    status = '200 OK'
    headers = [('Content-type', 'text/plain'),]

    with request:
        output = ''.join(request.content)

    return response(status, headers, [output])
Content filters
class Filter(object):
    def __init__(self, source, filter):
        self.source = source
        self.filter = filter
    def __iter__(self):
        with self.source:
            for data in self.source.content:
                 yield self.filter(data)

def application(request, response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain'),]

    return response(status, headers,
            Filter(request, str))
Middleware
def middleware(application):

  def _middleware(request, response):
    response = application(request, response)

    headers = list(itertools.dropwhile(
      lambda h: h[0].lower() == 'content-length',
      response.headers))

    return response(headers=headers,
      content=Filter(response, lambda x: 2*x))

  return _middleware

application = middleware(application)
Content length is painful

def application(request, response):

    status = '200 OK'
    headers = [('Content-type', 'text/plain'),]
    output = 'Hello world!'

    return response(status, headers,
                    [output], len(output))
Hiding the pain

def middleware(application):

  def _middleware(request, response):
    response = application(request, response)

    return response(
      content=Filter(response, lambda x: 2*x),
      length=None)

  return _middleware

application = middleware(application)
I decline your request!
def declined(request, response):
    return response

def middleware(*applications):
    def _middleware(request, response):
        for application in applications:
            interim = application(request, response)
            if interim is not response:
                return interim
        else:
            return response
    return _middleware

application = middleware(declined, application)
Is it a lost cause?
Is this all too late?

Is anyone interested interested in WSGI 2.0
anymore?

Although an interesting API, it cannot be
done on top of WSGI as changes how one
interacts with the web server.

So, try and do this and it does need to
supplant WSGI 1.0.
PyCon US 2012 - State of WSGI 2

Contenu connexe

Tendances

An Introduction to Tornado
An Introduction to TornadoAn Introduction to Tornado
An Introduction to TornadoGavin Roy
 
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 MinutesDjangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 MinutesNina Zakharenko
 
Powerful and flexible templates with Twig
Powerful and flexible templates with Twig Powerful and flexible templates with Twig
Powerful and flexible templates with Twig Michael Peacock
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionJoshua Thijssen
 
Learn flask in 90mins
Learn flask in 90minsLearn flask in 90mins
Learn flask in 90minsLarry Cai
 
Ansible not only for Dummies
Ansible not only for DummiesAnsible not only for Dummies
Ansible not only for DummiesŁukasz Proszek
 
What happens in laravel 4 bootstraping
What happens in laravel 4 bootstrapingWhat happens in laravel 4 bootstraping
What happens in laravel 4 bootstrapingJace Ju
 
Building a real life application in node js
Building a real life application in node jsBuilding a real life application in node js
Building a real life application in node jsfakedarren
 
Dance for the puppet master: G6 Tech Talk
Dance for the puppet master: G6 Tech TalkDance for the puppet master: G6 Tech Talk
Dance for the puppet master: G6 Tech TalkMichael Peacock
 
Real time server
Real time serverReal time server
Real time serverthepian
 
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-MechanizePerformance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanizecoreygoldberg
 
Javascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and GulpJavascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and GulpAll Things Open
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset
 
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...Innovecs
 

Tendances (20)

An Introduction to Tornado
An Introduction to TornadoAn Introduction to Tornado
An Introduction to Tornado
 
Django Celery
Django Celery Django Celery
Django Celery
 
Celery
CeleryCelery
Celery
 
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 MinutesDjangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
Djangocon 2014 - Django REST Framework - So Easy You Can Learn it in 25 Minutes
 
Powerful and flexible templates with Twig
Powerful and flexible templates with Twig Powerful and flexible templates with Twig
Powerful and flexible templates with Twig
 
Puppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 EditionPuppet for dummies - ZendCon 2011 Edition
Puppet for dummies - ZendCon 2011 Edition
 
Rails Performance
Rails PerformanceRails Performance
Rails Performance
 
Learn flask in 90mins
Learn flask in 90minsLearn flask in 90mins
Learn flask in 90mins
 
Celery with python
Celery with pythonCelery with python
Celery with python
 
Ansible not only for Dummies
Ansible not only for DummiesAnsible not only for Dummies
Ansible not only for Dummies
 
Flask – Python
Flask – PythonFlask – Python
Flask – Python
 
Intro to-puppet
Intro to-puppetIntro to-puppet
Intro to-puppet
 
What happens in laravel 4 bootstraping
What happens in laravel 4 bootstrapingWhat happens in laravel 4 bootstraping
What happens in laravel 4 bootstraping
 
Building a real life application in node js
Building a real life application in node jsBuilding a real life application in node js
Building a real life application in node js
 
Dance for the puppet master: G6 Tech Talk
Dance for the puppet master: G6 Tech TalkDance for the puppet master: G6 Tech Talk
Dance for the puppet master: G6 Tech Talk
 
Real time server
Real time serverReal time server
Real time server
 
Performance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-MechanizePerformance and Scalability Testing with Python and Multi-Mechanize
Performance and Scalability Testing with Python and Multi-Mechanize
 
Javascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and GulpJavascript TDD with Jasmine, Karma, and Gulp
Javascript TDD with Jasmine, Karma, and Gulp
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
 
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...
Reliable Python REST API (by Volodymyr Hotsyk) - Web Back-End Tech Hangout - ...
 

En vedette

How Plone's Security Works
How Plone's Security WorksHow Plone's Security Works
How Plone's Security WorksMatthew Wilkes
 
A look at FastCgi & Mod_PHP architecture
A look at FastCgi & Mod_PHP architectureA look at FastCgi & Mod_PHP architecture
A look at FastCgi & Mod_PHP architectureAimee Maree Forsstrom
 
Large platform architecture in (mostly) perl - an illustrated tour
Large platform architecture in (mostly) perl - an illustrated tourLarge platform architecture in (mostly) perl - an illustrated tour
Large platform architecture in (mostly) perl - an illustrated tourTomas Doran
 
A Brief Introduce to WSGI
A Brief Introduce to WSGIA Brief Introduce to WSGI
A Brief Introduce to WSGIMingli Yuan
 
Common gateway interface
Common gateway interfaceCommon gateway interface
Common gateway interfaceAnandita
 
FCGI, C++로 Restful 서버 개발
FCGI, C++로 Restful 서버 개발FCGI, C++로 Restful 서버 개발
FCGI, C++로 Restful 서버 개발현승 배
 

En vedette (10)

How Plone's Security Works
How Plone's Security WorksHow Plone's Security Works
How Plone's Security Works
 
A look at FastCgi & Mod_PHP architecture
A look at FastCgi & Mod_PHP architectureA look at FastCgi & Mod_PHP architecture
A look at FastCgi & Mod_PHP architecture
 
Large platform architecture in (mostly) perl - an illustrated tour
Large platform architecture in (mostly) perl - an illustrated tourLarge platform architecture in (mostly) perl - an illustrated tour
Large platform architecture in (mostly) perl - an illustrated tour
 
A Brief Introduce to WSGI
A Brief Introduce to WSGIA Brief Introduce to WSGI
A Brief Introduce to WSGI
 
Intro to PSGI and Plack
Intro to PSGI and PlackIntro to PSGI and Plack
Intro to PSGI and Plack
 
Common gateway interface
Common gateway interfaceCommon gateway interface
Common gateway interface
 
Common Gateway Interface
Common Gateway InterfaceCommon Gateway Interface
Common Gateway Interface
 
FCGI, C++로 Restful 서버 개발
FCGI, C++로 Restful 서버 개발FCGI, C++로 Restful 서버 개발
FCGI, C++로 Restful 서버 개발
 
CGI Presentation
CGI PresentationCGI Presentation
CGI Presentation
 
WSGI, Django, Gunicorn
WSGI, Django, GunicornWSGI, Django, Gunicorn
WSGI, Django, Gunicorn
 

Similaire à PyCon US 2012 - State of WSGI 2

Python magicmethods
Python magicmethodsPython magicmethods
Python magicmethodsdreampuf
 
Flask patterns
Flask patternsFlask patterns
Flask patternsit-people
 
How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30fiyuer
 
Osiąganie mądrej architektury z Symfony2
Osiąganie mądrej architektury z Symfony2 Osiąganie mądrej architektury z Symfony2
Osiąganie mądrej architektury z Symfony2 3camp
 
OSCON Google App Engine Codelab - July 2010
OSCON Google App Engine Codelab - July 2010OSCON Google App Engine Codelab - July 2010
OSCON Google App Engine Codelab - July 2010ikailan
 
Ejb3 Struts Tutorial En
Ejb3 Struts Tutorial EnEjb3 Struts Tutorial En
Ejb3 Struts Tutorial EnAnkur Dongre
 
Ejb3 Struts Tutorial En
Ejb3 Struts Tutorial EnEjb3 Struts Tutorial En
Ejb3 Struts Tutorial EnAnkur Dongre
 
Re-Design with Elixir/OTP
Re-Design with Elixir/OTPRe-Design with Elixir/OTP
Re-Design with Elixir/OTPMustafa TURAN
 
12 core technologies you should learn, love, and hate to be a 'real' technocrat
12 core technologies you should learn, love, and hate to be a 'real' technocrat12 core technologies you should learn, love, and hate to be a 'real' technocrat
12 core technologies you should learn, love, and hate to be a 'real' technocratJonathan Linowes
 
Rails vs Web2py
Rails vs Web2pyRails vs Web2py
Rails vs Web2pyjonromero
 
Why you shouldn’t edit silver stripe core files (and how to do it anyway)
Why you shouldn’t edit silver stripe core files (and how to do it anyway)Why you shouldn’t edit silver stripe core files (and how to do it anyway)
Why you shouldn’t edit silver stripe core files (and how to do it anyway)Loz Calver
 
TurboGears2 Pluggable Applications
TurboGears2 Pluggable ApplicationsTurboGears2 Pluggable Applications
TurboGears2 Pluggable ApplicationsAlessandro Molina
 
Advanced Python, Part 1
Advanced Python, Part 1Advanced Python, Part 1
Advanced Python, Part 1Zaar Hai
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8Alexei Gorobets
 
How To Test Everything
How To Test EverythingHow To Test Everything
How To Test Everythingnoelrap
 
Zend framework 03 - singleton factory data mapper caching logging
Zend framework 03 - singleton factory data mapper caching loggingZend framework 03 - singleton factory data mapper caching logging
Zend framework 03 - singleton factory data mapper caching loggingTricode (part of Dept)
 
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...
TorqueBox: The beauty of Ruby with the power of JBoss.  Presented at Devnexus...TorqueBox: The beauty of Ruby with the power of JBoss.  Presented at Devnexus...
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...bobmcwhirter
 

Similaire à PyCon US 2012 - State of WSGI 2 (20)

Python magicmethods
Python magicmethodsPython magicmethods
Python magicmethods
 
Flask patterns
Flask patternsFlask patterns
Flask patterns
 
Django Good Practices
Django Good PracticesDjango Good Practices
Django Good Practices
 
How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30How to disassemble one monster app into an ecosystem of 30
How to disassemble one monster app into an ecosystem of 30
 
Osiąganie mądrej architektury z Symfony2
Osiąganie mądrej architektury z Symfony2 Osiąganie mądrej architektury z Symfony2
Osiąganie mądrej architektury z Symfony2
 
OSCON Google App Engine Codelab - July 2010
OSCON Google App Engine Codelab - July 2010OSCON Google App Engine Codelab - July 2010
OSCON Google App Engine Codelab - July 2010
 
Ejb3 Struts Tutorial En
Ejb3 Struts Tutorial EnEjb3 Struts Tutorial En
Ejb3 Struts Tutorial En
 
Ejb3 Struts Tutorial En
Ejb3 Struts Tutorial EnEjb3 Struts Tutorial En
Ejb3 Struts Tutorial En
 
Re-Design with Elixir/OTP
Re-Design with Elixir/OTPRe-Design with Elixir/OTP
Re-Design with Elixir/OTP
 
Clean Code
Clean CodeClean Code
Clean Code
 
12 core technologies you should learn, love, and hate to be a 'real' technocrat
12 core technologies you should learn, love, and hate to be a 'real' technocrat12 core technologies you should learn, love, and hate to be a 'real' technocrat
12 core technologies you should learn, love, and hate to be a 'real' technocrat
 
Rails vs Web2py
Rails vs Web2pyRails vs Web2py
Rails vs Web2py
 
Why you shouldn’t edit silver stripe core files (and how to do it anyway)
Why you shouldn’t edit silver stripe core files (and how to do it anyway)Why you shouldn’t edit silver stripe core files (and how to do it anyway)
Why you shouldn’t edit silver stripe core files (and how to do it anyway)
 
TurboGears2 Pluggable Applications
TurboGears2 Pluggable ApplicationsTurboGears2 Pluggable Applications
TurboGears2 Pluggable Applications
 
Advanced Python, Part 1
Advanced Python, Part 1Advanced Python, Part 1
Advanced Python, Part 1
 
Dependency injection in Drupal 8
Dependency injection in Drupal 8Dependency injection in Drupal 8
Dependency injection in Drupal 8
 
How To Test Everything
How To Test EverythingHow To Test Everything
How To Test Everything
 
Zend framework 03 - singleton factory data mapper caching logging
Zend framework 03 - singleton factory data mapper caching loggingZend framework 03 - singleton factory data mapper caching logging
Zend framework 03 - singleton factory data mapper caching logging
 
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...
TorqueBox: The beauty of Ruby with the power of JBoss.  Presented at Devnexus...TorqueBox: The beauty of Ruby with the power of JBoss.  Presented at Devnexus...
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...
 
Play 2.0
Play 2.0Play 2.0
Play 2.0
 

Plus de Graham Dumpleton

Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.Graham Dumpleton
 
Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.Graham Dumpleton
 
“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.Graham Dumpleton
 
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.Graham Dumpleton
 
OpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaSOpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaSGraham Dumpleton
 
Automated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and KubernetesAutomated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and KubernetesGraham Dumpleton
 
PyCon HK 2015 - Monitoring the performance of python web applications
PyCon HK 2015 -  Monitoring the performance of python web applicationsPyCon HK 2015 -  Monitoring the performance of python web applications
PyCon HK 2015 - Monitoring the performance of python web applicationsGraham Dumpleton
 
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
PyCon AU 2015  - Using benchmarks to understand how wsgi servers workPyCon AU 2015  - Using benchmarks to understand how wsgi servers work
PyCon AU 2015 - Using benchmarks to understand how wsgi servers workGraham Dumpleton
 
PyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating DecoratorsPyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating DecoratorsGraham Dumpleton
 
PyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applicationsPyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applicationsGraham Dumpleton
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsGraham Dumpleton
 
DjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New RelicDjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New RelicGraham Dumpleton
 

Plus de Graham Dumpleton (13)

Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.Implementing a decorator for thread synchronisation.
Implementing a decorator for thread synchronisation.
 
Not Tom Eastman
Not Tom EastmanNot Tom Eastman
Not Tom Eastman
 
Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.Data analytics in the cloud with Jupyter notebooks.
Data analytics in the cloud with Jupyter notebooks.
 
“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.
 
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.
 
OpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaSOpenShift, Docker, Kubernetes: The next generation of PaaS
OpenShift, Docker, Kubernetes: The next generation of PaaS
 
Automated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and KubernetesAutomated Image Builds in OpenShift and Kubernetes
Automated Image Builds in OpenShift and Kubernetes
 
PyCon HK 2015 - Monitoring the performance of python web applications
PyCon HK 2015 -  Monitoring the performance of python web applicationsPyCon HK 2015 -  Monitoring the performance of python web applications
PyCon HK 2015 - Monitoring the performance of python web applications
 
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
PyCon AU 2015  - Using benchmarks to understand how wsgi servers workPyCon AU 2015  - Using benchmarks to understand how wsgi servers work
PyCon AU 2015 - Using benchmarks to understand how wsgi servers work
 
PyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating DecoratorsPyCon NZ 2013 - Advanced Methods For Creating Decorators
PyCon NZ 2013 - Advanced Methods For Creating Decorators
 
PyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applicationsPyCon US 2013 Making Apache suck less for hosting Python web applications
PyCon US 2013 Making Apache suck less for hosting Python web applications
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web Applications
 
DjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New RelicDjangoCon US 2011 - Monkeying around at New Relic
DjangoCon US 2011 - Monkeying around at New Relic
 

Dernier

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 

Dernier (20)

My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 

PyCon US 2012 - State of WSGI 2

  • 1. State of WSGI 2 Graham Dumpleton US PyCon March 2012 @GrahamDumpleton Graham.Dumpleton@gmail.com
  • 2. 1.0 (PEP 333) PEP created December 2003. A series of compromises. Iterable vs write(). Limited async capability.
  • 3. 1.0.1 (PEP 3333) PEP created September 2010. Clarifications to PEP 333 to handle str/bytes distinction in Python 3.X. Apache/mod_wsgi had already been providing support for Python 3.X since April 2009. Was a quick and painless process. NOT!
  • 4. Shortcomings Hard to write middleware correctly.
  • 5. Easy targets Drop start_response() and write(). def app(environ): status = '200 OK' content = 'Hello world' headers = [ ('Content-type', 'text/plain'), ('Content-length', str(len(content)))] return (status, headers, [content])
  • 6. Unwanted advances Support for async systems. The killer for async is that how you interact with wsgi.input means it is blocking. Hard to change it to make it work for async systems without making it too complicated for sync systems.
  • 7. Wishful thinking Standardised high level request/response objects. WebOb Werkzeug
  • 8. Interesting problems Cleaning up at end of request. Middleware and response content length. Unknown request content length. No compressed request content. No chunked requests. No full duplex HTTP.
  • 9. Has the boat sailed? Too much legacy code relying on WSGI 1.0. Potential missed opportunity for significant change when updating specification for Python 3.X. The possibilities of what could have been done are now probably only of academic interest.
  • 10. Ideas I toyed with Use context managers to improve resource management. Implementing 'wsgi.input' as an iterable. Support for chunked request content. Iterating through a list of applications until one agrees to handle it.
  • 11. Resource management Need to override close() of the iterable. Way too complicated for mere mortals. Makes wsgi.file_wrapper impotent.
  • 12. How complicated class Generator: def __init__(self, iterable, callback, environ): self.__iterable = iterable self.__callback = callback self.__environ = environ def __iter__(self): for item in self.__iterable: yield item def close(self): try: if hasattr(self.__iterable, 'close'): self.__iterable.close() finally: self.__callback(self.__environ)
  • 13. Not done yet class ExecuteOnCompletion: def __init__(self, application, callback): self.__application = application self.__callback = callback def __call__(self, environ, start_response): try: result = self.__application(environ, start_response) except: self.__callback(environ) raise return Generator(result, self.__callback, environ)
  • 14. Response as object Make response a context manager. Response content is an attribute. Consumer of response executes: with response: for data in response.content: process(data) Can override __enter__()/__exit__().
  • 15. Request as object Also make request a context manager. Request content is an attribute, but also make it an iterable. Reading all request content becomes: with request: content = ''.join(request.content) Content length becomes advisory only, so can implement chunked content and mutating input filters.
  • 16. Hello world def application(request, response): status = '200 OK' headers = [('Content-type', 'text/plain'),] output = 'Hello world!' return response(status, headers, [output])
  • 17. Consuming input def application(request, response): status = '200 OK' headers = [('Content-type', 'text/plain'),] with request: output = ''.join(request.content) return response(status, headers, [output])
  • 18. Content filters class Filter(object): def __init__(self, source, filter): self.source = source self.filter = filter def __iter__(self): with self.source: for data in self.source.content: yield self.filter(data) def application(request, response): status = '200 OK' headers = [('Content-type', 'text/plain'),] return response(status, headers, Filter(request, str))
  • 19. Middleware def middleware(application): def _middleware(request, response): response = application(request, response) headers = list(itertools.dropwhile( lambda h: h[0].lower() == 'content-length', response.headers)) return response(headers=headers, content=Filter(response, lambda x: 2*x)) return _middleware application = middleware(application)
  • 20. Content length is painful def application(request, response): status = '200 OK' headers = [('Content-type', 'text/plain'),] output = 'Hello world!' return response(status, headers, [output], len(output))
  • 21. Hiding the pain def middleware(application): def _middleware(request, response): response = application(request, response) return response( content=Filter(response, lambda x: 2*x), length=None) return _middleware application = middleware(application)
  • 22. I decline your request! def declined(request, response): return response def middleware(*applications): def _middleware(request, response): for application in applications: interim = application(request, response) if interim is not response: return interim else: return response return _middleware application = middleware(declined, application)
  • 23. Is it a lost cause? Is this all too late? Is anyone interested interested in WSGI 2.0 anymore? Although an interesting API, it cannot be done on top of WSGI as changes how one interacts with the web server. So, try and do this and it does need to supplant WSGI 1.0.

Notes de l'éditeur

  1. \n
  2. Back in time, web servers capable of running Python web applications used different interfaces. Web applications had to support more than one if wanted to be portable. Original WSGI PEP created late 2003 with the goal of standardising on one interface. Different approaches though meant it ended up being a series of compromises.\n
  3. Although not ideal, the WSGI specification achieved its goal. Then Python 3 came along though. Python 3 was a problem because of change to make default string type be Unicode. After much teeth pulling and unwelcome distractions, a revised PEP was produced to affirm how WSGI should work with Python 3.\n
  4. What were the problems with WSGI? Apart from various ambiguities and corner cases that weren't really addressed, one of the bigger issues was that writing WSGI middleware that was correct was hard. This is what in part has driven the idea that we need a WSGI 2.0.\n
  5. The main suggestion that arises to make WSGI simpler is to remove the duplication in having a write() callback as well as being able to return an iterable. Do away with write() and the next step is to remove start_response(). An application therefore returns a tuple alone, consisting of status line, headers and iterable for content.\n
  6. Conceptually not too hard, but every time there was discussion about improving WSGI, people would step up with their own wish list for some specific requirement they had. The loudest of these was those pushing for full support for async systems. Reality is that a simple interface which supports both sync and async is hard.\n
  7. The other idea that kept popping up was standardised high level request/response objects. The difficulty here is that you are introducing something that would likely need to keep evolving, which is a bad thing to have in a specification because you are really forcing people to one implementation rather than to a specification.\n
  8. Personally I thought there were a range of other aspects to WSGI which were more fundamental where we could do better. These include resource management, ensuring output filters don't consume/generate too much data and inability to have certain types of request content or mutating input filters.\n
  9. So, there are simple changes that could be made and there are also deeper changes we could make to try and address some of the shortcomings and make it somewhat better. The big question now though is whether anyone cares and whether it is all just too late.\n
  10. I had my own ideas for changes that could be made to WSGI. Not having a lot of time and being burned out by the whole WSGI on Python 3 discussions, I have refrained from pushing them. Some of those ideas I played around with though are these listed, but there are others I have had.\n
  11. It is perhaps not used often and so is of academic interest, but being able to specify code which runs on completion of a request is just way too hard. It is not sufficient to do it when application returns. It has to be done when the iterable is consumed.\n
  12. To do this requires wrapping the existing iterable and in that wrapper implementing a close() method. One immediate problem with that is that it means that wsgi.file_wrapper as an optimisation for serving up files no longer works, as the server doesn't see the original iterable type.\n
  13. If we actually want this to work as a generic middleware rather than force a change to the application, then we also need to provide a wrapper for the application.\n
  14. Back when WSGI was created, nice ways of executing code at completion of a task didn't really exist, thus why close() method was used. Since then though we got context managers. If we have a response object where content is an attribute, then we can make it a context manager as well, allowing both entry and exit actions to be specified.\n
  15. Turns out that one can also do the same thing for request as well. If we are going to make such a change, lets be more radical though and make access to request content be an iterable just like with a response. If we make content length advisory only, then we can support chunked requests and mutating input filters.\n
  16. Using objects for both request and response a hello world applications becomes this. The arguments are the request object, but also a response object. The request object has as attributes the environ dictionary and content. The response object is used to created the response, but can also be used for more than that.\n
  17. As mentioned before, request content length is advisory only and wouldn't need to be used to read request content. You would only check it if generating 413 response or to decide how you might handle content. Using iterable for request content does mean need new cgi.FieldStorage, or use a file object like wrapper.\n
  18. Using iterables for both request and response content now means we have symmetry. You can have one type of filter that can be applied to both. One could even technically send request content direct back out to a client via a filter.\n
  19. Middleware is still possible. The status line, headers and content are just attributes of the response, but being an object now, we can simplify things by allowing one to make a call against the response to substitute just parts of it and create a new response object. Dealing with content length though is still very painful.\n
  20. We can make this simpler though by separating content length out and treating it as a special attribute. Being an attribute, the iterator for content can then enforce it, only returning the specified amount of data when consuming the content.\n
  21. For our middleware, if we don't know what the resulting content length might be after filter applied, we can just set it to None. That or we set it to the new known length. Do note, it is not the intent to create a full on request/response object. We are only embodying the absolute minimum to enforce correct protocol and usage.\n
  22. As well as starting to use the request/response object to enforce correct usage, the way they are passed around can be used to implement new behaviour. In mod_python a handler could decline a request and subsequent one used instead. Paste had this as Cascade middleware, but we can make that cleaner using response as a marker.\n
  23. So could this be an improvement to WSGI? Does anyone care anymore? It isn't possible to do it on top of WSGI as intent is to open up the server interface to features that can't be done with WSGI now. If you are interested, then come talk to me. History means I am quite wary of opening this one up on the WEB-SIG list.\n
  24. If you do come and talk to me, let it be known that I don't care what colour it is. Arguments over what parameters are called or how they are composed together are largely not important. I want us to solve the underlying issues I can see rather than whether it looks pretty and there are more issues than just what I have covered here.\n