Rapid web development using tornado web and mongodb

Rapid Web Development with
Tornado Web and MongoDB
Ikai Lan
Twitter: @ikai

Friday, June 10, 2011

About the speaker

• Developer Relations at Google
• Software Engineering background
• Lives in San Francisco, CA
• Writes Java, Python on a day to day basis
• Twitter: @ikai


This talk

• Why Tornado Web and MongoDB?
• Whirlwind tour of Tornado (get it?!)
• Intro to MongoDB
• Brief look at a demo app


What I won’t be talking
about

• Scalability - loaded topic
• Reliability - another loaded topic


So why tornado/
mongo?


Where does this joy
come from?


Freedom.

So really, my talk should
have been called “The
Joy of Tornado Web and
Mongo DB”


Intangibles

• Tornado - fast development, very few rules
• MongoDB - freedom from a predeﬁned
schema
• Python, of course. Dynamically typed
variables, duck typing plus everything else
that is cool about the language


Tornado Web history
• Open Sourced by Facebook
• ... formerly of FriendFeed
• ... former Googlers (that’s why the
framework looks like App Engine’s webapp
- they wrote it)
• Tornado Web powers FriendFeed


A whirlwind tour

Tornado features
• Out of the box auth with Twitter,
Facebook, Google and FriendFeed
• Out of box support for localized strings
• Simple templating that allows arbitrary
Python code
• Asynchronous requests
• Small codebase

Simple handlers
import tornado.ioloop
import tornado.web

class HelloHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world :)")

class GoodbyeHandler(tornado.web.RequestHandler):
def get(self):
self.render("goodbye.html",
message=”Goodbye, world”)

application = tornado.web.Application([
(r"/hello", HelloHandler),
(r"/goodbye", GoodbyeHandler),
])

if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()


Asynchronous nature
class MainHandler(tornado.web.RequestHandler):

@tornado.web.asynchronous
def get(self):
http = tornado.httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed-api.com/v2/feed/bret",
callback=self.on_response)

def on_response(self, response):
if response.error: raise tornado.web.HTTPError(500)
json = tornado.escape.json_decode(response.body)
self.write("Fetched " + str(len(json["entries"])) + " entries "
"from the FriendFeed API")
self.finish()


Unrestrictive templating
{% for student in [p for p in people if p.student and p.age > 23] %}
<li>{{ escape(student.name) }}</li>
{% end %}

# Sample of arbitrary functions in templates
def add(x, y):
return x + y
template.execute(add=add)

### The template
{{ add(1, 2) }}


Lies, damn lies and benchmarks


Saving data

MongoDB is an object
database

• No schema - anything that can be a JSON
object can be stored
• You can deﬁne new collections on the ﬂy


Getting started with
Mongo in 4 steps
• Download a binary for your system
• Create a data directory (mkdir -p /data/db)
• Run mongod
• Install pymongo
• ... start writing code!


Pymongo code sample
connection = pymongo.Connection()
database = connection["your_database"]

def get_or_create_location_by_id(location_id):
"""
Attempts to fetch location data from database. If it doesn't exist,
create it. Note that this is NOT concurrency safe.
"""
location_data = database["locations"].find_one({ "_id" : location_id })
if location_data is None:
location_data = { "_id" : location_id,
"guards": [],
"owner" : None,
"history" : [],
"last_extort_time" : None
}
database.location.save(location_data, safe=True)
return location_data


Pymongo queries
# Add Tyson as a guard to every location owned by “Joe Smith”
locations = database["locations"].find({ "owner" : "Joe Smith" })
for location in locations:
location["guards"].append("Tyson")
database["locations"].save(location)

# Find everyone who has Tyson as a guard
locations = database["locations"].find({"guards" : "Tyson"})

# Find everyone who has *ONLY* Tyson as a guard
locations = database["locations"].find({"guards" : ["Tyson"]}) # Note []s

# Find everyone who has Tyson as a guard whose owner is “Ikai Lan”
locations = database["locations"].find({"guards" : "Tyson",
"owner" : "Ikai Lan" })


Mold as you go along

Freedom from

• ... waiting to get started
• ... predeﬁning a schema
• ... worrying about complex data structures
that don’t ﬁt well into the SQL box.
Anything that is JSON can be stored!


More MongoDB
features
• Simple Geospatial indexing
• GridFS - distributed ﬁlesystem running on
MongoDB
• Out of the box mapreduce
• Out of the box replication, data partitioning


Putting it all together
• MobSquare, a location based game that
uses the Facebook API.
• Mashup: Maﬁa Wars + FourSquare
• https://github.com/ikai/mobsquare-demo
• “Check in” to locations, take them over,
extort money and ﬁght other gangs


Why are Tornado/
Mongo a ﬁt?
• If I haven’t said it enough yet, they’re fun
• Facebook API performance varies -
asynchronous so we can serve requests
while waiting
• Highly structured player data


OAuth 2 upgrade ﬂow
class OnLoginHandler(tornado.web.RequestHandler):

@tornado.web.asynchronous
def get(self):
# Store this somewhere
code = self.get_argument("code")
access_token_url = ACCESS_TOKEN_URL_TPL + code
client = httpclient.AsyncHTTPClient()
client.fetch(access_token_url, self.on_fetched_token)

def on_fetched_token(self, response):
""" Callback inokved when the auth_token is fetched """
matches = ACCESS_TOKEN_REGEX.search(response.body)
if matches:
access_token = matches.group(1)
client = httpclient.AsyncHTTPClient()
# lambda is effectively a function factory for us
client.fetch(API["profile"] % access_token,
lambda response: self.on_profile_fetch(response, access_token))

def on_profile_fetch(self, response, access_token):
""" Callback invoked when we have fetched the user's profile """
profile = json.loads(response.body)
profile["access_token"] = access_token
profile_id = db.save_profile(profile)
self.set_secure_cookie("user_id", str(profile_id))
self.redirect("/") # implictly calls self.finish()


Known gotchas

• Immaturity of frameworks, tools
• Tornado templating errors result in
somewhat useless stack traces
• MongoDB nuances
• Tornado community fairly small


Questions?

• Ikai Lan - @ikai on Twitter
• Github: https://github.com/ikai
• Google Proﬁle: https://proﬁles.google.com/
ikai.lan/about
• Thank you for having me at Pycon!


Attributions

• Slide 6: Kirstin Jennings - “big smile!” http://www.flickr.com/photos/methyl_lives/2973265796/

• Slide 8: Tony Blay - “I am a bird now!” http://www.flickr.com/photos/toniblay/59415205/

• Slide 12: Antonio Sofi - “whirlwind” http://www.flickr.com/photos/webgol/36546734

• Slide 17: Tornado Web documentation - http://www.tornadoweb.org/documentation

• Slide 18: Christine of Robot Brainz - “Cassette” http://www.flickr.com/photos/robotbrainz/
2786347158/

• Slide 23: Samuel Stroube - “Play-Doh on a Picnic Table” http://www.flickr.com/photos/samoube/
203586664


Rapid web development using tornado web and mongodb

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (7)

Similaire à Rapid web development using tornado web and mongodb

Similaire à Rapid web development using tornado web and mongodb (20)

Plus de ikailan

Plus de ikailan (14)

Dernier

Dernier (20)

Rapid web development using tornado web and mongodb