2. Introduction
• Virtual Machines
• az-1.region-a.geo-1
• web-created
• Ubuntu 11.04 64 bit
• 3+ VM’s / model
• total 20 VM’s
• Benchmark Suite
• byte-unixbench
• mbw
• iozone
• iperf
• pgbench
• Hadoop wordcount
Mediu XXLar
XSmall Small Large XLarge
m ge
vCPU 1 2 2 4 4 8
• Data Filtering
MEM (GB) 1 2 4 8 16 32
• best VM / model
DISK (GB) 30 60 120 240 480 960
• average by 10
Price ($/hr) 0.04 0.08 0.16 0.32 0.64 1.28
3. byte-unixbench
4500
Si ngl e Thr ead
4000
M t i Thr ead
ul
3500
3000
2500
2000
1500
1000
500
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• byte-unixbench index measures overall system performance
• in multi-thread testing, n-Thread = n-vCPU
• system with the same amount of vCPU exhibits similar performance
• memory size does not have much impact on performance
• 2 x vCPU => 1.5 x performance
4. mbw
12000
10000
M CPY
EM
8000 DM
UP
MCBLOCK
6000
4000
2000
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• mbw 128
• MB/s
• different systems exhibit similar memory performance
5. iozone – os disk
7000000
w i te
r
6000000 r ew i t e
r
5000000 r andom w i t e
r
r ead
4000000 r er ead
r andom r ead
3000000
2000000
1000000
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• iozone -Mcew -i0 -i1 -i2 -s4g -r256k -f /io.tmp
• KB/s
• difference systems exhibit similar write performance
• L / XL / XXL systems exhibit much better read performance
• cgroup blkio throttling? QEMU blk throttle? Different disk types?
6. iozone – data disk
6000000
w i te
r
5000000 r ew i t e
r
r andom w i t e
r
4000000 r ead
r er ead
3000000 r andom r ead
2000000
1000000
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• iozone -Mcew -i0 -i1 -i2 -s4g -r256k -f /mnt/io.tmp
• KB/s
• difference systems exhibit similar write performance
• XL / XXL systems exhibit much better read performance
• cgroup blkio throttling? QEMU blk throttle? Different disk types?
7. iperf
XXLarg
XSmall Small Medium Large XLarge
e
XSmall 25 25 25 25 25 25
Small 25 50 50 50 50 50
Medium 25 50 100 100 100 100
Large 25 50 100 200 200 200
XLarge 25 50 100 200 400 400
XXLarge 25 50 100 200 400 650
• (x, y) represents the bandwidth between two systems
• Mbps
• bandwidth limited by the system with lower configuration
• cisco quantum plugin?
8. hadoop wordcount single no
de
900
800 2GB
700
600
500
400
300
200
100
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• hadoop wordcount application provided in official distribution
• test directory with 3 files, total file size is 2 GB.
• test result shows the time needed to finish the calculation (s)
9. hadoop wordcount multiple nod
es
1000
900
800
700
600
500
400
300
200
100
0
1 x 2 x 3 x 4 x Sm l
al XXLar ge
Xsm l
al XSm l
al XSm l
al XSm l
al
• dfs.replication = nNodes
• test directory with 3 files, total file size is 2 GB.
• test result shows the time needed to finish the calculation (s)
10. pgbench
1800
1600 Si ngl e Thr ead
M t i Thr ead
ul
1400
1200
1000
800
600
400
200
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• postgresql-9.1, postgresql-contrib-9.1
• pgbench -i -s 16 pgbench
• pgbench -t 2000 -c 16 –j n -U postgres pgbench
• in multi-thread testing, n-Thread = n-vCPU
11. defects – pgbench single thr
ead
1200
1000
800 N m
or al
D ect
ef
600
400
200
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• defects were observed in all VM models
• test results were smooth on the same VM instance
• the following test results were not affected on defected instances
• mbw
• iperf
• byte-unixbench
12. defects – iozone write result
s
300000
N m
or al
250000 D ect
ef
200000
150000
100000
50000
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• test performed on OS disks only
• write performance seems to be the major problem
13. defects – iozone read result
s
7000000
N m
or al
6000000
D ect
ef
5000000
4000000
3000000
2000000
1000000
0
XSm l
al Sm l
al M um
edi Lar ge XLar ge XXLar ge
• test performed on OS disks only
• read performance is similar for all instances in both cases
14. defect rate
7
= %
35
20
• 7 defected instances were found out of 20 total instances
• defect rate too high for deploying production systems
• need extra caution when VM’s are auto-generated by API’s
15. conclusion
先以欲勾牵，后令入佛智。
鸠摩罗什大师译 《维摩诘所说经 . 佛道品第八 》
• HP defects were not directly related to OpenStack
• OpenStack still lacks key functionalities for production deployment
• building IaaS service is more complicated than installing OpenStack
• open source IaaS software => IaaS support and service => $$$
