Christian Westbrook / CoTweet (cw@cotweet.com) at ErLounge SF/Bay Jan. 12th, 2010 @ Pier 38 http://meetup.com/erlounge

1. ErLounge/SF Bay
2010.1.12

2. Welcome

3. hello_world.erl

4. wello_horld.erl

5. wello_horld.erl

6. wello_horld.erl

7. CoTweet Robot: Task

8. CoTweet Robot: Task
• Pull updates from Twitter API:
• n * 10,000 channels ( n is going up -- o/ )

9. CoTweet Robot: Task
• Pull updates from Twitter API:
• n * 10,000 channels ( n is going up -- o/ )
channel update req. 4 http requests on average
= n million http requests / hr

10. CoTweet Robot: Task
• Pull updates from Twitter API:
• n * 10,000 channels ( n is going up -- o/ )
• Minimize latency:
• Ideally < 300s between updates

11. CoTweet Robot: Task
• Pull updates from Twitter API:
• n * 10,000 channels ( n is going up -- o/ )
• Minimize latency (< 300s between updates)
• <blink>Survivable</blink> !

12. Primordial Ooze: Ruby
Note: not a real robot

13. Primordial Ooze: Ruby

14. Primordial Ooze: Ruby
Like any tool, rails is great for some things...

16. Robot Evolution: I

17. Robot Evolution: I
• Main process: receive/after loop:
service_loop() ->
spawn(?MODULE, do_work, []),
receive after SleepyTimeMS * 1000 -> ok end,
service_loop().

18. Robot Evolution: I
• Main process: receive/after loop
• Reads user info from database
• For every N records ...

19. Robot Evolution: I
• Main process: receive/after loop
• Reads user info from database
• For every N records:
• spawn() a worker to process bucket:
• Retrieve updates from Twitter API
• Insert into database (returning ID)
• Insert (using ID) into memcached

20. spawn() responsibly.
• I <3 supervisors
• trap_exit + PID = spawn_link(M, F, [A]).
• {PID,Ref} = spawn_monitor(M,F,[A]).

21. Single Assignment != Stateless

22. Single Assignment != Stateless
• gen_server, gen_event, gen_fsm: State
-record(State, {foo,bar}).
handle_call({set_foo, V}, _From, State) ->
{reply, State#state{ foo = V };

23. Single Assignment != Stateless
• ets
• Many concurrent reads? No problem.
start() ->
ets:new(state, [named_table, public]).
set_foo(V) ->
ets:insert(state, {foo,V}).

24. Single Assignment != Stateless
• ets
• Many concurrent reads? No problem.
• Many concurrent writes?
• May need to Serialize through a gen_server.
handle_call({set_foo, V}, _From, State) ->
ets:insert(state, {foo, V}),
{reply, State};

25. Single Assignment != Stateless
• gen_server, gen_event, gen_fsm: State
• ets: app-wide
• serialize writes through a gen_server
• mnesia: persistent
• process dictionary: put(K,V) / get(K)
• Tiny, transient, process-level
• (think: private):

26. Robot Evolution: II

27. Robot Evolution: II
• All state held in ets
• Robot comprises many ‘services:’
• Feeder: poll db as needed, update ets
• Scheduler: add tasks to work queue
• Business logic:
• even work load, limit resource(s)
• Dispatcher: service queue for workers

28. Hm, our “workers” may block waiting for
result(i.e., ID) of an insert from DB . . .

29. Hm, our “workers” may block waiting for
result(i.e., ID) of an insert from DB . . .
Needs more OTP.

30. Robot Evolution: III
• gen_event as simplest MUX ever
• no time for downtime: hot deployment

31. gen_event
• Extraordinarily easy to multiplex:
• Create an event manager: gen_event
• Add handlers
• Fire an event
• Not just for logging . . .

32. Deployment
-module(my_app).
-export([reload_modules/0]).
-define(APP_MODS, [my_app, my_service]).
reload_modules() ->
lists:map(fun(M) -> code:load_file(M) end,
?APP_MODS).
See: http://www.erlang.org/cgi-bin/ezmlm-cgi/4/36236

34. Open Source
“Standing on the
shoulders of giants...”
-- Bob Ippolito
S. Mestrom with K. Newby

35. Open Source
• plists (parallel list operations)
• lhttpc (much lighter than inets)
• rfc_4627 (utf8 JSON decoding)
• epgsql (PostGreSQL driver)
• erlang_twitter (Twitter API via xmerl)
• merle (erlang memcached adapter)
• mochi (congrats to MochiMedia!)

36. Thanks
• SocialMedia -- Ken Thom
• CoTweet
• You

37. ErLounge
• Grab another drink
• Share your stories