Tofu is an image processing system developed by COOKPAD that generates thumbnail images on demand from original images stored in S3. Previously, COOKPAD would generate all thumbnail sizes upfront, using significant storage. Tofu addresses this by using a Apache module called mod_tofu that generates thumbnails from original images only when requested. This on-demand approach is faster, more scalable, and saves storage by not pre-generating all sizes. Tofu leverages the scalability of AWS to process large volumes of image resizing with high performance.

1. tofu
COOKPAD’s image system
Issei Naruta
COOKPAD Inc.
2 Nov 2011

2. about me
Issei Naruta ( )
Engineer, COOKPAD Inc.
Infrastructure Team
image system / high performance web
twitter: @mirakui

3. cookpad.com
900 million PV/month
Over 1 million recipes
Rails 3
AWS(Amazon Web Services)

4. COOKPAD.has_many(:thumbnails)
280x280
144x216
original image
(1000x1000)

5. When/How do we
generate thumbnails?

6. BEFORE TOFU (a year ago)
User Generate thumbnails
uploads
an image
app (rails)
original thumbnails
storage

7. Problem
We should generate thumbnails of
all existing images as new size
when another size thumbnails are needed
(We have 7 million original images!)

8. AFTER TOFU (now)
User Store original image
uploads
an image
app (rails)
original original
storage

9. AFTER TOFU (now)
Store original image Generate thumbnails
On demand
tofu
original
storage

10. tofu architecture
akamai (cdn)
ELB
Apache Apache Apache
(mod_tofu) (mod_tofu)
… (mod_tofu)
S3

11. mod_tofu.so
C (Apache module)
Apache
ImageMagick

12. thumbnail url
category thumbnail size
http://hostname/recipes/1/280/0e473ffbce0eb1e28a892fe4086b65b6.jpg
id
SHA1(category+id+size+SECRET_WORD)

13. resizing
height=free
width=280
/recipes/1/280/0e473ffbce0eb1e28a892fe4086b65b6.jpg

14. cropping
100 x 100
& center cropping
/recipes/1/100x100c/819d7abe2a8b9261aa3c43e948abe7b3.jpg

15. jpeg quality (for mobile)
low ‘100x100cq10’
‘100x100cq30’
‘100x100cq50’
‘100x100cq70’
‘100x100cq90’
high

16. On-demand resizing by url
Fast
Scalable

17. On-demand resizing by url
Fast
Scalable

18. Peak traffic
akamai (cdn) 7000 req/sec
ELB 800 req/sec
EC2 c1.xlarge
Apache Apache Apache
(mod_tofu) (mod_tofu)
… (mod_tofu)
(8cores)
8 instances
100 req/sec
per instance
S3

19. fetch images
from S3
resize by
ImageMagick

20. On-demand resizing by url
Fast
Scalable

21. tofu scalability = AWS scalability
akamai (cdn)
autoscaling
ELB
more instances, by itself
more capacity
Apache Apache Apache
(mod_tofu) (mod_tofu)
… (mod_tofu)
Oh yeah
S3 It has a huge
capacity!

22. tofu effect
Our prototyping workflow became more fast
and flexible
Save storage diskspace
Maintenance becomes easily

23. Thank you for your listening