1. A Virtual World Distributed Server
developed in Erlang as a Tool for analysing Needs
of Massively Multiplayer Online Game Servers
Erlang/OTP User Conference
Stockholm on November 10, 2005
Michał Ślaski
michal@erlang-consulting.com
© 2005 Erlang Training and Consulting Ltd

2. Contents
• Massively Multiplayer Online Games
• Problems to solve
• Prototyping
• Using Erlang/OTP for the prototype
• Client and server applications
• Determining the maximum number of players
• Architectures with single and double-function nodes
• Synchronisation of the game’s state
• Conclusions
Page 2 © 2005 Erlang Training and Consulting Ltd

3. Massively Multiplayer Online Games
• Significant increasing in number of active subscribers
• Fast growing online games market
9 000 000
8 000 000
Total active subscribers
7 000 000
6 000 000
5 000 000
4 000 000
3 000 000
2 000 000
1 000 000
0
1997 1998 1999 2000 2001 2002 2003 2004 2005
Source: MMOGCHART.COM Source: IGDA, quot;2004 Persistent Worlds White Paperquot;
Page 3 © 2005 Erlang Training and Consulting Ltd

4. Problems to solve
• Big amount of data to process
• Frequent synchronisation
– acceptable delay 150 - 500 ms
• Adaptation to the connection quality
• Limited opportunity for full testing with thousands of
players during early phase of development
Page 4 © 2005 Erlang Training and Consulting Ltd

5. Prototyping
• Experimenting with the functional
and technical aspects
• Determining the scalability of the architecture
• Testing new solutions
Page 5 © 2005 Erlang Training and Consulting Ltd

6. Using Erlang/OTP for the prototype
• Open source
• Compatible with various platforms
• Concurrent oriented programming
– one process for every connected player
• Efficient internal communication
• Supporting distributed applications
• Distributed database Mnesia
• Distributed load testing tool IDX-Tsunami
Page 6 © 2005 Erlang Training and Consulting Ltd

7. Client application
• Java + Java3D
• Actions:
– moving
– collecting objects
– magic
– chat
• Player’s state
kept on the server
Page 7 © 2005 Erlang Training and Consulting Ltd

8. Server application
• Server distributed on a
• The game terrain
cluster of machines
divided into zones
• Players placed within the
same zone are processed
on the same node
Page 8 © 2005 Erlang Training and Consulting Ltd

9. Determining the maximum number of users
1 200
Legend
Collecting object transaction
1 050
transaction time [msec]
900
750
600
450
180
300
160
overload
150 122 players
140
number of players
The server
0 120
0 20 40 60 80 100 120 140 160
was
time of the test [s] 100
overloaded 80
800
in the 60th
overload 60
second of
700
network traffic [kbits / sec]
40
the test
600
20
500 0
0 20 40 60 80 100 120 140 160
400
time of the test [s]
300
200
100 Legend
Outgoing network traffic
0
0 20 40 60 80 100 120 140 160
time of the test [s]
Page 9 © 2005 Erlang Training and Consulting Ltd

10. Architectures with double-function nodes
900
800
700
number of players
600
500
400
300
200
Legend
maximum number of players that c an be handled by the arc hitec ture
100
0 1 2 3 4 5 6 7 8 9 10
number of nodes processing network traffic and player's actions
Page 10 © 2005 Erlang Training and Consulting Ltd

11. Architectures with single-function nodes
700 120
600 100
CPU utilisation [%]
500 80
number of players
400 60
300 40
200 20
Legend
maximum number of players
ac tion proc essing nodes' CPU utilisation
network proc essing nodes' CPU utilisation
100 0
0 1 2 3 4 5 6
number of nodes processing player's actions (2 network nodes)
Page 11 © 2005 Erlang Training and Consulting Ltd

12. Synchronising issue
Client A Server Client B
Try to pick up a mushroom
Try to pick up a mushroom
Action status - OK
mushroom collected
by client A
Action status - FAIL
Page 12 © 2005 Erlang Training and Consulting Ltd

13. Desynchronisation of the game’s state
0,1
• Server Legend
suc c essful ac tions
0,09
number of transactions / number of players
not suc c essful ac tions
is not y=0.04855 (suc c essful)
y=0.0009119 (not suc c essful)
0,08
overloaded
0,07
• More 0,06
successful 0,05
actions
0,04
0,03
0,02
0,01
0
0 100 200 300 400 500 600 700 800 900 1 000
time of the test [s]
Page 13 © 2005 Erlang Training and Consulting Ltd

14. Desynchronisation of the game’s state
0,05
• Server is Legend
suc c essful ac tions
number of transactions / number of players
0,045
not suc c essful ac tions
overloaded y=0.04855 (suc c essful)
y=0.0009119 (not suc c essful)
0,04
0,035
• More
actions 0,03
fail 0,025
0,02
0,015
0,01
0,005
0
0 100 200 300 400 500 600 700 800 900 1 000
time of the test [s]
Page 14 © 2005 Erlang Training and Consulting Ltd

15. Conclusions
• Erlang/OTP is appropriate for developing prototypes
of the distributed MMOG systems
• Rapid development of the prototype allows
examining different algorithms in a relatively short
period of time
• Future work
– algorithms not dependent on the player’s position
Page 15 © 2005 Erlang Training and Consulting Ltd

16. Contents
• Massively Multiplayer Online Games
• Problems to solve
• Prototyping
• Using Erlang/OTP for the prototype
• Client and server applications
• Determining the maximum number of players
• Architectures with single and double-function nodes
• Synchronisation of the game’s state
• Conclusions
Page 16 © 2005 Erlang Training and Consulting Ltd