Async programming and python

1. Async
Programming and
Python
PyCon India, 2014
Chetan Giridhar

2. Basics
• Programming tasks:
o I/O bound
o CPU bound
• Say, you’re doing I/O
o Will it complete immediately? When will it be done?
o Wont they block you?

3. Blocking I/O: Example
import requests
r = requests.get(‘http://google.co.in’)
r.status_code
• What if the request takes a long time?
• Operation blocks until all the data is recieved from the server
Can we do something in the meanwhile?
Can we run another task, concurrently?

4. Non Blocking / Async
• Non-blocking means the ability to make continuous
progress at all times
• Resources needed for a response must not be
monopolized
• As such it can enable both lower latency, higher
throughput

5. Programming Models
Synchronous model
time
time
time
Threaded model
Asynchronous model
Task 1 Task 2 Task 3

6. Math
• Task = make a call to http://ip.jsontest.com
• Say, Task = Task1 = Task2 = Task 3 = 400ms
• Sync Model
o Time taken = Task1+ Task2 + Task3 = 1.2 sec
• Threaded Model
o Time taken = 510 ms
• Async Model
o Time taken = 460 ms
What’s the
magic here?

7. Async Paradigm
• Clients requests the event driven web server;
• requests are processed by event loop;
• event handlers cater to events with callbacks
Client Event driven server I/O loop
Event driven I/O loop
Request
IO loop handles
request
Event Handlers

8. Reactor Pattern
• Typical non blocking frameworks work on a
philosophy of single threaded event loop
o keeps polling for events
Reactor
Pattern
Waiting for Events
Handling Events

9. More Details!
• Framework typically maintains a list of file
descriptors(fd), events to monitor and
corresponding event handlers for each of the fd
• Listening to events on a fd is a kernel space task
o epoll, [kqueue/select] – libraries provide event notifications in a non-blocking
way
• Epoll watches file descriptors (sockets) and returns
needed (READ, WRITE & ERROR) events

10. Async way
• Async strategy aims for:
o Making I/O tasks non blocking
o I/O tasks would run independently
o generate an event when tasks are complete
o with help of callbacks
• Benefits
o No need to wait till blocking I/O tasks are complete
o More responsive real time applications
o Thread safety isn't an issue
• Can we solve any other Python problems with this
mechanism?
o Eliminating GIL?

11. Async in Python
• Frameworks
o Tornado
o Twisted
o Gevent
• Modules
o Tulip
o Asyncio

12. Async in Python
• Frameworks
o Tornado
o Twisted
o Gevent
• Modules
o Tulip
o Asyncio

13. Asyncio
• Part of Python library
o The latest module for async application development
• Only for Python > 3.4
o Incompatible with prior versions
• A whole new way to development
o Let’s you write self contained, synchronous looking tasks
o Run two infinite loops at the same time on the same thread
• Works with other framework
o Tornado, Twisted, GEvent

14. Asyncio…
• Write single threaded concurrent code
• Principle of Interleaved execution of subroutines
• Co-operative scheduling
o Only one task at a time
• Based on libevent
o Select, kpoll, kqueue

15. Asyncio: Components
Event loop
Co-routines, Futures, Tasks
Transports, Protocols

16. Asyncio: Components
• Event loop
o Register, executing and cancelling calls
o Schedule execution of a task (co-routine)
o Creates transport (async client and server)
o Runs I/O callbacks (Watches file descriptors)
o Thread interface
o [BaseEventLoop.create_task()] or async()
o [asyncio.get_event_loop()]

17. Asyncio: Components
• Co-routine
o Generator (“yield from”)
o suspended at preset execution points, and
o resumed later by keeping track of local state
o @coroutine decorator

18. Asyncio: Components
• Task
o responsible for executing a coroutine
o If coroutine yields from a future, the task suspends the execution of the
coroutine and waits for the future
o coroutine restarts when future is done
o Subclass of class Future
o [async(coroutine)]
o BaseEventLoop.create_task(coro)

19. Asyncio: Components
• Future
o A class
o for results that are
available later
import asyncio
@asyncio.coroutine
def slow_operation(future):
yield from asyncio.sleep(1) <- Co-routine suspend
future.set_result('Future is done!')
def got_result(future):
print(future.result())
loop.stop()
loop = asyncio.get_event_loop() <- Event loop
future = asyncio.Future() <- Future object
asyncio.async(slow_operation(future)) <- Task
future.add_done_callback(got_result)
try:
loop.run_forever()
finally:
loop.close()

20. Asyncio: Components
• transport
o represent connections such as sockets, SSL connection and pipes
o Async socket operations
• Usually frameworks implement e.g. Tornado
• protocols
o represent applications such as HTTP client/server, SMTP, and FTP
o Async http operation
o [loop.create_connection()]

21. Example: Asyncio Redis
import asyncio
import asyncio_redis
@asyncio.coroutine
def my_subscriber(channels):
connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
subscriber = yield from connection.start_subscribe()
yield from subscriber.subscribe(channels)
while True:
reply = yield from subscriber.next_published()
print('Received: ', repr(reply.value), 'on channel', reply.channel)
loop = asyncio.get_event_loop()
asyncio.async(my_subscriber('channel-1'))
asyncio.async(my_subscriber('channel-2'))
loop.run_forever()

22. Example: Asyncio ‘Tasks’
import asyncio
@asyncio.coroutine
def factorial(name, number):
f = 1
for i in range(2, number+1):
print("Task %s: Compute factorial(%s)..." % (name, i))
yield from asyncio.sleep(1)
f *= i
print("Task %s: factorial(%s) = %s" % (name, number, f))
loop = asyncio.get_event_loop()
tasks = [
asyncio.async(factorial("A", 2)),
asyncio.async(factorial("B", 3)),
asyncio.async(factorial("C", 4))]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

23. Async in Python
• Frameworks
o Tornado
o Twisted
o Gevent
• Modules
o Tulip
o Asyncio

24. Tornado Async
• Event Loop => tornado.ioloop
• Coroutine => tornado.gen.coroutine
• Future => tornado.concurrent.future
• Transport/Protocol => tornado.iostream
• Bridge the gap => tornado.platform.asyncio –
Combines asyncio and tornado in same event loop

25. Tornado Async Http
import tornado.ioloop
from tornado.httpclient import AsyncHTTPClient
def handle_request(response):
'''callback needed when a response arrive'''
if response.error:
print("Error:", response.error)
else:
print(’Success')
print(response.body)
Before Event Loop Starts!
Success
b'{"ip": "117.192.252.80"}n'
Callback
http_client = AsyncHTTPClient() # initialize http client
http_client.fetch(” http://ip.jsontest.com/", handle_request)
print("Before Event Loop Starts!")
tornado.ioloop.IOLoop.instance().start() # start the tornado ioloop

26. Tornado Coroutine
import tornado.web
import tornado.gen
from tornado.httpclient import AsyncHTTPClient
class GenAsyncHandler(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
response = yield http_client.fetch("http://google.com")
print(response)
application = tornado.web.Application([ (r"/",
GenAsyncHandler), ])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
gen.coroutine schedules the generator
to be resumed when the Future is
resolved
‘yield’ makes the function a generator
The generator in turn returns a Future
instance
In this case, response, will resolve with
response from fetch or an exception

27. Tornado Engine
class MainHandlerAsync(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.engine
def get(self):
req = tornado.httpclient.HTTPRequest("http://127.0.0.1:8888/",)
client = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(client.fetch, req)
self.finish()
application = tornado.web.Application([
(r"/async", MainHandlerAsync),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()

28. Let’s create our Future!
myfuture.py server.py
import time
import datetime
from tornado.concurrent import return_future
class AsyncHTTPClient(object):
@return_future
def fetch(self, url, callback=None):
print("In my fetch")
time.sleep(0.02)
result = str(datetime.datetime.utcnow())
callback(result)
import tornado.web
import tornado.gen
from myfuture import AsyncHTTPClient
def test(arg):
print('In test:' + arg)
class GenAsync(tornado.web.RequestHandler):
@tornado.gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
r = yield http_client.fetch(“http://google.com”,test)
print(r)
application = tornado.web.Application([
(r"/", GenAsync),])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()

29. Performance:
Blocking vs Async

30. Work to be achieved

31. Performance Results
ab -n 500 -c 10 http://localhost:8888/blocking
ab -n 500 -c 10 http://localhost:8888/async
6000
5000
4000
3000
2000
1000
0
Time per request
Async Blocking
Async
Blocking
200
150
100
50
0
Requests per second
Async Blocking
Async
Blocking

32. Learnings
• Async programming is an efficient, easy to
understand design and code
• Python asyncio module is comprehensive
• Has generic use cases for vast variety of
applications
o Responsive web applications
o Networking applications
• Requires a new way to program and design

33. Recommendations
• Async programming is not a holistic solution
• It has its own pros and cons
o Suitable for primarily I/O bound applications
o Needs enough tasks available to run
• asyncio module is only available for Python 3
applications
• Also explore other methods of concurrency:
o Eventlets
o STM
o Multiprocessing/threads
o Special languages e.g. GO, Scala
• Understand and use 

34. References
• asyncio – http://python.org
• Python asyncio –
o http://www.buzzcapture.com
o www.slideshare.net/saghul
• Tornado – http://tornadoweb.org
• Multithreading – www.drdobbs.com
• Event loop: https://docs.python.org/3/library/asyncio-eventloop.
html

35. Contact Us
• Chetan Giridhar
o www.technobeans.com
o https://github.com/cjgiridhar
• Vishal Kanaujia
o www.freethreads.wordpress.com
o https://github.com/vishalkanaujia

36. Backup

37. Asyncio: example
import asyncio
@asyncio.coroutine
def create():
yield from asyncio.sleep(3.0)
print("(1) create file")
@asyncio.coroutine
def write():
yield from asyncio.sleep(1.0)
print("(2) write into file")
@asyncio.coroutine
def close():
print("(3) close file")
@asyncio.coroutine
def test():
asyncio.async(create())
asyncio.async(write())
asyncio.async(close())
yield from asyncio.sleep(2.0)
loop.stop()
loop = asyncio.get_event_loop()
asyncio.async(test())
loop.run_forever()
print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop))
loop.close()

38. Python Async Modules
• Asynccore
• Asyncchat
• Gevent
• Twisted
• Eventlets

39. Concurrency Techniques
• Multithreading/processing
• Green Threads
• STM

40. Tornado + AsyncIO
from tornado.platform.asyncio import AsyncIOMainLoop
from tornado.httpclient import AsyncHTTPClient
import asyncio
AsyncIOMainLoop().install() -- # Tell Tornado to use the asyncio eventloop
loop = asyncio.get_event_loop() -- # get the loop
http_client = AsyncHTTPClient() -- # the Tornado HTTP client
def aio_fetch(client, url, **kwargs):
fut = asyncio.Future()
client.fetch(url, callback=fut.set_result, **kwargs)
return fut
@asyncio.coroutine
def main():
print("fetching my site")
mysite = yield from aio_fetch(http_client, "http://google.com/")
print("hello httpbin")
httpbin = yield from aio_fetch(http_client, "http://httpbin.org?q=%d" % mysite.code)
print(httpbin.body)
loop.run_until_complete(main())

41. Co-routine v/s Callback
import asyncio
def just_print_messages(loop):
print('Just print')
loop.call_later(1, just_print_messages, loop)
def main():
loop = asyncio.get_event_loop()
try:
loop.call_soon(just_print_messages, loop)
loop.run_forever()
finally:
loop.close()
if __name__ == '__main__':
main()
import asyncio
@asyncio.coroutine
def just_print_messages():
while True:
print('Just print')
yield from asyncio.sleep(1)
def main():
loop = asyncio.get_event_loop()
try:
loop.run_until_complete(just_print_messages())
finally:
loop.close()
if __name__ == '__main__':
main()

42. Async in NodeJS
request is an
var http = require(‘http’);
event
var server = http.createServer;
server.on(‘request’, function(request,response) {
response.writeHead(200);
response.end(‘Hello World’);
}).listen(8001);
Callback
console.log(‘Server running on port 8001’);