Misultin (pronounced mee-sool-téen) is an Erlang library for building fast lightweight HTTP(S) servers, which also supports websockets. https://github.com/ostinelli/misultin

1. Misultin
[pron.: Mee-sool-téen]
An Erlang library for building fast lightweight HTTP(S) servers
>-|-|(°>
Erlang User Group London - May 16th, 2011

2. Who am I?
Technologist and Entrepreneur. I am
particularly interested in Erlang, Python,
scalability, research, development and
integration of the multiple technologies
behind the creation of ubiquitous
computing. I’m currently co-founder
and CTO of WideTag Inc [widetag.com].
e: roberto@ostinelli.net
t: @ostinelli

3. <1>

4. What are Misultins?
Misultin are a culinary specialty of the Lake of Como.
They are lake fishes that go by the latin name of Alosa
fallax lacustris, treated and cooked in a traditional
and specific way.

5. What is Misultin?
Misultin (pronounced mee-sool-téen) is an Erlang
library for building fast lightweight HTTP(S) servers,
which also supports websockets.
>-|-|(°>

6. Why another lib?
When do programmers start a new lib?
‣ Nothing similar exists
‣ Different Approach / Focus
‣ Total Control / Knowledge
‣ Experiment/Hack/Academic

7. Features v0.7.1
‣ Embedded, not application
‣ ~ HTTP/1.1
‣ Support for GET, POST, HEAD, PUT, DELETE, TRACE and CONNECT
‣ SSL
‣ Output compression [gzip, deflate]
‣ Chunked / Stream / File sending [attachment, visualization]
‣ Customizable parameters [max_connections, post_max_size, get_url_max_size, ...]
‣ Multiple servers on a single node [custom name / nameless]
‣ Websockets [also on SSL]
‣ Trapping of client closing a browser in Comet applications
‣ With / without Parametrized Modules

8. </1>

9. <2>

10. Code | “Hello World”
Simple “Hello World” using parametrized modules.

11. Code | “Hello World”
Simple “Hello World” not using parametrized modules.

12. Code | File Sending
Simple file sending.

13. Code | REST

14. Code | Websocket
1. Start Misultin specifying a Websocket loop.

15. Code | Websocket
2. Send the HTML + Js page.

16. Code | Websocket
3. Handle Websocket Data on server.

17. Code | Websocket
Live Demo.

18. Code | Server Options
start_link(Options) -> Result
Options = [Option]
Option = {ip, list() | tuple()} |
{port, integer()} |
{name, atom()|false} |
{max_connections, integer()} |
{post_max_size, integer()} |
{get_url_max_size, integer()} |
{compress, true|false} |
{loop, fun()} |
{autoexit, true|false} |
{ws_loop, fun()|none} |
{ws_autoexit, true|false} |
{backlog, integer()} |
{acceptors_poolsize, integer()} |
{recv_timeout, integer()} |
{ssl, SslOptions}
SslOptions = [SslOption(§)]

19. </2>

20. <3>

21. Design | Overview
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
Acceptor
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process Http Process Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

22. Design | Crash: Acceptor
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

23. Design | Crash: Acceptor
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

24. Design | Crash: Acceptor
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
'EXIT' message: . erlang:spawn/1
gen_server respawns . gen_tcp:controlling_process/2
a new acceptor . call to gen_server to:
Main gen_server
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

25. Design | Crash: HttpProcess
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

26. Design | Crash: HttpProcess
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
1.
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

27. Design | Crash: HttpProcess
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
1.
Http Process HTTP Logic
erlang:spawn_link/1
2.
HttpLoop Application Logic

28. Design | Crash: HttpProcess
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
3. 'DOWN' message:
1.
decrease open
connections counter
Http Process HTTP Logic
erlang:spawn_link/1
2.
HttpLoop Application Logic

29. Design | Crash: HttpLoop
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

30. Design | Crash: HttpLoop
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

31. Design | Crash: HttpLoop
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
'EXIT' message:
HttpProcess sends
HTTP 500 error erlang:spawn_link/1
message to client
HttpLoop Application Logic

32. Design | Crash: gen_server
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

33. Design | Crash: gen_server
1.
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

34. Design | Crash: gen_server
1. 2.
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
- erlang:monitor/2: for gen_server count
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

35. Design | Crash: gen_server
1. 2.
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
terminate(_,_) -> - erlang:monitor/2: for gen_server count
exit(HttpPid, kill). 3.
Http Process HTTP Logic
erlang:spawn_link/1
HttpLoop Application Logic

36. Design | Crash: gen_server
1. 2.
proc_lib:spawn_link/3
gen_server
Acceptor TCP Acceptor
[misultin]
. erlang:spawn/1
. gen_tcp:controlling_process/2
Main gen_server . call to gen_server to:
- http_pid_ref_add/2: add new pid to ref
terminate(_,_) -> - erlang:monitor/2: for gen_server count
exit(HttpPid, kill). 3.
Http Process HTTP Logic
erlang:spawn_link/1
4.
HttpLoop Application Logic

37. </3>

38. <4>

39. Benchmarks | Foreword
‣ No such thing as generic benchmarks: specific case
[fast application times, loads of connections, small data]
‣ ‘Fast’ is only 1 of the ‘n’ features you want
[stability, features, ease of maintenance, low standard deviation, code usability, ...]
‣ This is no ‘pissing’ contest. *Not* what I wanted to see:

40. Benchmarks | Test
‣ Not only Erlang
‣ Chosen libraries are all lightweight & dynamic
[compare apple with apples: no Yaws, no Nginx]
‣ Periodically close down the TCP connection
‣ All tests run on a single CPU
‣ No RAM/CPU considerations

41. Benchmarks | Test
‣ ‘Dynamic’ tests: no “Hello world”
The test basically asks servers to:
• check if a GET variable is set
• if the variable is not set, reply with an XML stating the error:
<http_test><error>no value specified</error></http_test>
• if the variable is set, echo it inside an XML
<http_test><value>MYVALUE</value></http_test>

42. Benchmarks | Setup
‣ Patched HttPerf
[for higher file handling]
‣ Virtualized Up-to-date Ubuntu 10.04 LTS, 2CPU, 1.5G Ram
[tuning of /etc/sysctl.conf and /etc/security/limits.conf]
‣ Libraries:
• Misultin 0.7.1 (Erlang R14B02)
• Mochiweb 1.5.2 (Erlang R14B02)
• Cowboy master 420f5ba (Erlang R14B02)
• NodeJS 0.4.7
• Tornadoweb 1.2.1 (Python 2.6.5)
‣ All the libraries have been run with the standard settings. Erlang
was launched with Kernel Polling enabled, and with SMP
disabled so that a single CPU was used by all the libraries.

43. Benchmarks | Results

44. Benchmarks | Results

45. Benchmarks | Results

46. Benchmarks | Results

47. </4>

48. <5>

49. Comet | Practical issues
HTTP Logic
Http Process
spawn_link/1
HttpLoop
Application Logic

50. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
spawn_link/1
HttpLoop
Application Logic

51. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
Actual TCP
sending
spawn_link/1
Data to be Sent
HttpLoop
Application Logic

52. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
Actual TCP
sending => monitor Socket while in socket_loop?
spawn_link/1
Data to be Sent
HttpLoop
Application Logic

53. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
Actual TCP
sending => monitor Socket while in socket_loop?
spawn_link/1 => get data from Socket while in socket_loop?
Data to be Sent
HttpLoop
Application Logic

54. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
Actual TCP
sending => monitor Socket while in socket_loop?
spawn_link/1 => get data from Socket while in socket_loop?
Data to be Sent CONFLICT:
• Catch Client’s Socket closed event
HttpLoop • Support HTTP pipelining
Application Logic

55. Comet | Practical issues
HTTP Logic
Http Process
1. Building Req: {active, once} | gen_tcp:recv/3
2. Spawn HttpLoop, enter socket_loop/3
Actual TCP
sending => monitor Socket while in socket_loop?
spawn_link/1 => get data from Socket while in socket_loop?
Data to be Sent CONFLICT:
• Catch Client’s Socket closed event
HttpLoop • Support HTTP pipelining
SOLUTION:
Application Logic
handle_http(Req) ->
Req:options([{comet, true}]),
...

56. </5>

58. Thank you
e: roberto@ostinelli.net
t: @ostinelli
>-|-|(°>