Caching Techniques in Python: Memoization, Invalidation and Process Level Cache

Caching techinques in
python
Michael Domanski
europython 2010

czwartek, 22 lipca 2010

who I am

• python developer, professionally for a few
years now
• experienced also in c and objective-c
• currently working for 10clouds.com


Interesting intro

• a bit of theory
• common patterns
• common problems
• common solutions


How I think about
cache

• imagine a giant dict storing all your data
• you have to manage all data manually
• or provide some automated behaviour


similar to....

• manual memory managment in c
• cache is memory
• and you have to controll it manually


proﬁts

• improved performance
• ...?


problems

• managing any type of memory is hard
• automation often have to be done custom
each time


common patterns


memoization


• very old pattern (circa 1968)
• we own the name to Donald Mitchie


how it works

• we assosciate input with output, and store
in somewhere
• based on the assumption that for a given
input, output is always the same


code example
CACHE_DICT = {}

def cached(key):
def func_wrapper(func):
def arg_wrapper(*args, **kwargs):
if not key in CACHE_DICT:
value = func(*args, **kwargs)
CACHE_DICT[key] = value
return CACHE_DICT[key]
return arg_wrapper
return func_wrapper


what if output can
change?

• our pattern is still usefull
• we simply need to add something


cache invalidation


There are only two hard problems in Computer
Science: cache invalidation and naming things
Phil Karlton


• basically, we update data in cache
• we need to know when and what to
change

• the more granular you want to be, the
harder it gets


code example
def invalidate(key):
try:
del CACHE_DICT[key]
except KeyError:
print "someone tried to invalidate not present
key: %s" %key


common problems


invalidating too much/
not enough

• ﬂushing all data any time something changes
• not ﬂushing cache at all
• tragic effects


@cached('key1')
def simple_function1():
return db_get(id=1)

@cached('key2')
return db_get(id=2)

# SUPPOSE THIS IS IN ANOTHER MODULE

@cached('big_key1')
def some_bigger_function():
"""
this function depends on big_key1, key1 and key2
"""
def inner_workings():
db_set(1, 'something totally new')
#######
## imagine 100 lines of code here :)
######
inner_workings()

return [simple_function1(),simple_function2()]

if __name__ == '__main__':
simple_function1()
simple_function2()
a,b = some_bigger_function()
assert a == db_get(id=1), "this fails because we didn't invalidated cache properly"


invalidating too soon/
too late

• your cache have to be synchronised to you
db
• sometimes very hard to spot
• leads to tragic mistakes


@cached('key1')
return db_get(id=1)

@cached('key2')
return db_get(id=2)


db_set(1, 'something')
value = simple_function1()
db_set(2, 'something else')
#### now we know we used 2 cached functions so....
invalidate('key1')
invalidate('key2')
#### now we know we are safe, but for a price
return simple_function2()

if __name__ == '__main__':
some_bigger_function()


superposition of
dependancy
• somehow less obvious problem
• eventually you will start caching effects of
computation
• you have to know very preciselly of what
your data is dependant


@cached('key1')
return db_get(id=1)

@cached('key2')
return db_get(id=2)


@cached('key')

return {
'1': simple_function1(),
'2': simple_function2(),
'3': db_get(id=3)
}

if __name__ == '__main__':
simple_function1()
# somewhere else
db_set(1, 'foobar')
# and again
db_set(3, 'bazbar')
invalidate('key')
# ooops, we forgot something
data = some_bigger_function()
assert data['1'] == db_get(id=1), "this fails because we didn't manage to invalidate all the
keys"


summing up
• know your data....
• be aware what and when you cache
• take care when using cached data in
computation


common solutions


process level cache


why?

• very fast access
• simple to implement
• very effective as long as you’re using single
process


clever tricks with dicts


invalidation


application level cache


memcache


• battle tested
• scales
• fast
• supports a few cool features
• behaves a lot like dict
• supports time-based expiration

libraries?

• python-memcache
• python-libmemcache
• python-cmemcache
• pylibmc


why no benchmarks

• not the point of this talk :)
• benchmarks are generic, caching is speciﬁc
• pick your ﬂavour, think for yourself


code example
cache = memcache.Client(['localhost:11211'])

def memcached(key):
def func_wrapper(func):
value = cache.get(str(key))
if not value:
cache.set(str(key), value)
return value
return arg_wrapper
return func_wrapper


code example
def mem_invalidate(key):
cache.set(str(key), None)


batch key managment


• what if I don’t want to expire each key
manually

• that’s a lot to remember
• and we have to be carefull :(


groups?

• group keys into sets
• which are tied to one key per set
• expire one key, instead of twenty


how to get there?

• store some extra data
• you can store dicts in cache
• and cache behaves like dict
• so it’s a case of comparing keys and values


#we start with specified key and group
key='some_key'
group='some_group'

# now retrieve some data from memcached
data=memcached_client.get_multi(key, group)
# now data is a dict that should look like
#{'some_key' :{'group_key' : '1234',
# 'value' : 'some_value' },
# 'some_group' : '1234'}
#
if data and (key in data) and (group in data):
if data[key]['group_key']==data[group]:
return data[key]['value']


def cached(key, group_key='', exp_time=0 ):

# we don't want to mix time based and event based expiration models
if group_key : assert exp_time==0, "can't set expiration time for grouped keys"
def f_wrapper(func):
value = None
if group_key:
data = cache.get_multi([tools.make_key(group_key)]+[tools.make_key(key)])
data_dict = data.get(tools.make_key(key))
if data_dict:
value = data_dict['value']
group_value = data_dict['group_value']
if group_value != data[tools.make_key(group_key)]:
value = None
else:
value = cache.get(key)
if not value:
if exp_time:
cache.set(tools.make_key(key), value, exp_time)
elif not group_key:
cache.set(tools.make_key(key), value)
else: # exp_time not set and we have group_keys
group_value = make_group_value(group_key)
data_dict = { 'value':value, 'group_value': group_value}
cache.set_multi({ tools.make_key(key):data_dict, tools.make_key(group_key):group_value })
return value
arg_wrapper.__name__ = func.__name__
return arg_wrapper
return f_wrapper


questions?


code samples @
http://github.com/
mdomans/europython2010


follow me

twitter: mdomans
blog: blog.mdomans.com


Caching Techniques in Python: Memoization, Invalidation and Process Level Cache

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Similar to Caching Techniques in Python: Memoization, Invalidation and Process Level Cache

Similar to Caching Techniques in Python: Memoization, Invalidation and Process Level Cache (12)

Recently uploaded

Recently uploaded (20)

Caching Techniques in Python: Memoization, Invalidation and Process Level Cache