1. Running memcached clusters on Amazon EC2 In The Brain of Daniel Sikar Skills Matter – London, UK 01.12.2010

3. Daniel Sikar Consultant Developer Trainer Internet & scientific data acquisition applications Runs Amazon Web Services, Oracle Coherence and Intro to Hadoop courses at Skills Matter [email_address]

5. LCD memcached? High performance memory object caching system Clusters? Group of computers working together AWS? Infrastructure as a Service

6. Caching

8. CPU cache

9. Database query cache

10. Caching... Used to decrease latency.

11. Clusters

13. Reliability

14. Scalability

15. Clusters... If a unit falls off, the cluster continues behaving the same way. Units can be added and subtracted as needed.

16. Clusters... The cluster functions as a whole.

17. Infrastructure as a Service

19. On demand

20. Pay as you go

21. EC2

22. Amazon Machine Image (AMI) > Instance > EC2 Key Concepts I

23. One AMI can launch one or more instances EC2 Key Concepts II

25. $ wget http://(..)ec2-api-tools.zip # unpack to /usr # config env vars $ env | grep EC2 (...) EC2 tools

26. EC2 tools $ ls /usr/bin/ec2* $ ls /usr/lib/ec2* $ ec2ver $ ec2dim -a > amis.txt $ cat amis.txt | grep <...>

27. $ ec2auth -h $ ec2addgrp -h $ ec2dgrp -h EC2 – Security Groups (firewall)

28. $ ec2addgrp dbgroup -d &quot;db security group&quot; $ ec2addgrp mcgroup -d &quot;mc security group&quot; $ ec2addgrp wbgroup -d &quot;wb security group&quot; EC2 – Security Groups (firewall)

29. EC2 – Security Groups (firewall)

30. $ ec2auth dbgroup -o wbgroup -u <account id> $ ec2auth wbgroup -o dbgroup -u <account id> $ ec2auth wbgroup -o mcgroup -u <account id> $ ec2auth mcgroup -o wbgroup -u <account id> EC2 – Security Groups (firewall)

31. EC2 – Security Groups (firewall)

32. $ ec2auth dbgroup -p 3306 -s 0.0.0.0/0 $ ec2auth wbgroup -p 80 -s 0.0.0.0/0 $ ec2auth wbgroup -p 22 -s 0.0.0.0/0 $ ec2auth dbgroup -p 22 -s 0.0.0.0/0 $ ec2auth mcgroup -p 22 -s 0.0.0.0/0 $ ec2dgrp dbgroup mcgroup wbgroup EC2 – Security Groups (firewall)

33. AMI Instance Running EC2 Instances

34. Key Pairs

35. Generating a key pair $ ec2addkey aws_key > ~/.ssh/aws_key.pem $ chmod 600 ~/.ssh/aws_key.pem

36. Running EC2 instances $ ec2run ami-f4340180 -g dbgroup -t t1.micro -z eu-west-1a -k aws_key # plus one web and one memcached server $ ssh -i ~/.ssh/aws_key.pem <user>@<dns>

37. Configuring a database server $ sudo apt-get install mysql server $ sudo vim /etc/mysql/my.cnf #bind-address = 127.0.0.1 bind-address = 0.0.0.0 # notice caching configuration option $ sudo /etc/init.d/mysql restart $ mysql -u root -p mysql> create user 'dbuser'@'%' identified by <pwd> mysql> grant all privileges on *.* to 'dbuser'@'%' # add some data

39. Configuring a memcached server $ ssh -i ~/.ssh/aws_key.pem <user>@<dns> $ sudo apt-get update $ sudo apt-get install memcached $ memcached -h | head $ ps aux | grep memcached $ sudo vim /etc/memcached.conf # mods $ sudo /etc/init.d/memcached restart $ ps aux | grep memcached

41. PHP

42. Java

43. Python

44. Ruby

45. Perl

47. PostgreSQL

48. Erlang

49. Lua

50. Lisp

51. Cold Fusion

52. OCaml

53. (...)

54. Running a memcache(d) php client $ sudo apt-get install php5-memcached $ sudo apt-get install libmemcached-dev $ sudo apt-get install php5-dev php-pear $ sudo apt-get install php5-mysql $ sudo pecl install memcache $ sudo vim /etc/php5/apache2/php.ini extension=memcache.so $ sudo /etc/init.d/apache2 restart $ wget http://localhost/phpinfo.php $ cat phpinfo.php | grep memcache # copy files

55. What about the memcache cluster?

56. AWS workflow

57. Creating a memcached AMI $ ec2din (...) <volume id> (...) $ ec2addsnap <volume id> (...) <snapshot id> (...) $ ec2dim ami-f4340180 $ ec2reg -s <snapshot id> --kernel <kernel id> -n &quot;name&quot; -d &quot;desc&quot; $ ec2dim <ami id> $ ec2run <ami id> -g mcgroup -t t1.micro -z eu-west-1a -k aws_key -n 2 # add to cluster & test

58. Tidying up $ ec2kill <instance id>

60. Conclusion Setting up and running memcached clusters is easy on IAAS environments such as Amazon EC2

61. Questions?

62. Coming up next: Intro to Hadoop 20.12.2010 Amazon Web Services 24.01.2011 Skills Matter – London, UK

63. Thank you for attending! Running memcached clusters on Amazon EC2 In The Brain of Daniel Sikar Skills Matter – London, UK 01.12.2010