YCSB

1. www.vmcd.org
基于 mongodb 的压力评测工具 YCSB 的一些概括
YCSB 是一款基于 NOSQL Benchmark 的工具,提供了多种方式模拟测试 nosql 的负载,基于现在对 nosql 的一些压力测试还没有得到重视.
YCSB 的全面的测试功能可以给即将上线的 nosql 提供一种另类保障。
There are many new serving databases available, including:
 PNUTS
 BigTable
 HBase
 Hypertable
 Azure
 Cassandra
 CouchDB
 Voldemort
 MongoDb
 OrientDB
 Infinispan
 Dynomite
 Redis
 GemFire
 GigaSpaces XAP
 DynamoDB
 Couchhase
 Aerospike

2. www.vmcd.org
下面的数据仅供参考,测试于虚拟机服务器。
给一个简单的 load data 的例子：
[root@mysqlstd ycsb-0.1.4]# ./bin/ycsb load mongodb -P workloads/workloada -p mongodb.url=mongodb://127.0.0.1:27017 -p mongodb.database=newdb -p
mongodb.writeConcern=normal -s >data
Loading workload...
Starting test.
0 sec: 0 operations;
10 sec: 18448 operations; 1837.08 current ops/sec; [INSERT AverageLatency(us)=423.35]
20 sec: 42134 operations; 2366.71 current ops/sec; [INSERT AverageLatency(us)=373.44]
30 sec: 61185 operations; 1904.34 current ops/sec; [INSERT AverageLatency(us)=661.58]
40 sec: 85308 operations; 2411.09 current ops/sec; [INSERT AverageLatency(us)=324.83]
50 sec: 97785 operations; 1247.2 current ops/sec; [INSERT AverageLatency(us)=985.33]
50 sec: 100000 operations; 2662.26 current ops/sec; [INSERT AverageLatency(us)=371.24]

3. www.vmcd.org
load data 之后 可以开始模拟压测了.YCSB 主要分为以下几种模式：
Workload A: Update heavy workload
This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.
Workload B: Read mostly workload
This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags.
Workload C: Read only
This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).
Workload D: Read latest workload
In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status
updates; people want to read the latest.
Workload E: Short ranges

4. www.vmcd.org
In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each
scan is for the posts in a given thread (assumed to be clustered by thread id).
Workload F: Read-modify-write
In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records
are read and modified by the user or to record user activity.
其中 E 模式的 short range 存在问题,当然我们也可以自定义模式:
# Yahoo! Cloud System Benchmark
# Workload A: Update heavy workload
# Application example: Session store recording recent actions
#
# Read/update ratio: 50/50
# Default data size: 1 KB records (10 fields, 100 bytes each, plus key)
# Request distribution: zipfian
recordcount=100000
operationcount=100000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=true

5. www.vmcd.org
readproportion=1
updateproportion=0
scanproportion=0
insertproportion=0
修改 read update insert 的百分比来达到模拟真实环境的目的, 下面给个压测例子
比如压测纯读操作:
[root@mysqlstd ycsb-0.1.4]# ./bin/ycsb run mongodb -P workloads/workloadc -P large.dat -p mongodb.url=mongodb://127.0.0.1:27017 -p
mongodb.database=newdb -p mongodb.writeConcern=normal -s >data
Loading workload...
Starting test.
0 sec: 0 operations;
10 sec: 49375 operations; 4922.24 current ops/sec; [READ AverageLatency(us)=192.56]
18 sec: 100000 operations; 6141.57 current ops/sec; [READ AverageLatency(us)=159.72]
95%读+%5 写：
[root@mysqlstd ycsb-0.1.4]# ./bin/ycsb run mongodb -P workloads/workloadd -P large.dat -p mongodb.url=mongodb://127.0.0.1:27017 -p
mongodb.database=newdb -p mongodb.writeConcern=normal -s >data
Loading workload...
Starting test.

6. www.vmcd.org
0 sec: 0 operations;
10 sec: 43497 operations; 4333.23 current ops/sec; [INSERT AverageLatency(us)=633.66] [READ AverageLatency(us)=196.33]
20 sec: 92795 operations; 4925.37 current ops/sec; [INSERT AverageLatency(us)=792.15] [READ AverageLatency(us)=167.74]
21 sec: 100000 operations; 5637.72 current ops/sec; [INSERT AverageLatency(us)=379.57] [READ AverageLatency(us)=163.45]
另外 thumbtack 对 YCSB 进行了修改,增强了一些功能 开源地址：
https://github.com/thumbtack-technology/ycsb
主要增加支持了 Aerospike and Couchbase 目前 Aerospike 已经开源 针对 SSD 进行了专门优化 ：
http://www.aerospike.com/blog/entrepreneurs-break-all-the-rules-aerospike-goes-open-source/
同时 MongoDB 的驱动程序从 2.8.0 版（在 Mongo 2.2 发布之前的版本）升级到了 2.10.1 版,并实现了对 readPreference 配置的设置
mongodb.readPreference = primary|primaryPreferred|secondary|secondaryPreferred

7. www.vmcd.org
下面结合 thumbtack 修改的版本做个测试 可以得到具体数据:
[root@mysqlstd ycsb]# fab ycsb_load:db=mongodb
[10.0.32.38] Executing task 'ycsb_load'
2014-07-15 01:09:00-07:00
[10.0.32.38] run: echo "/root/ycsb/bin/ycsb load mongodb -s -p mongodb.url=mongodb://127.0.0.1:27017 -p
workload=com.yahoo.ycsb.workloads.CoreWorkload -p updateretrycount=1000 -p mongodb.writeConcern=normal -p mongodb.database=ycsb -p

8. www.vmcd.org
recordcount=5000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=32 -p insertretrycount=10 -p
readretrycount=1000 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p operationcount=2400000000 -p fieldnameprefix=f -p maxexecutiontime=2400
-p mongodb.readPreference=primaryPreferred -p measurementtype=timeseries -p reconnectiontime=1000 -p fieldlength=10 -p insertstart=0 -p
insertcount=5000000 > /root/ycsb/2014-07-15_01-09_mongodb_load.out 2> /root/ycsb/2014-07-15_01-09_mongodb_load.err" | at 01:09 today
[10.0.32.38] out: job 13 at 2014-07-15 01:09
Done.
Disconnecting from 10.0.32.38... done.
[mongo@mysqlstd ~]$ /data/mongodb/mongodb/bin/mongo
MongoDB shell version: 2.6.1
connecting to: test
> show dbs
admin 0.031GB
local 0.031GB
newdb 0.500GB
newdb1 0.500GB
newdb2 0.500GB
ycsb 1.500GB
> use ycsb
switched to db ycsb
> db.usertable.c
db.usertable.clean( db.usertable.convertToCapped( db.usertable.copyTo( db.usertable.createIndex(
db.usertable.constructor db.usertable.convertToSingleObject( db.usertable.count(
> db.usertable.count()
2675710

9. www.vmcd.org
模拟压力环境：workload mode=A
[root@mysqlstd ycsb]# fab ycsb_run:db=mongodb,workload=A
[10.0.32.38] Executing task 'ycsb_run'
2014-07-15 02:13:00-07:00
[10.0.32.38] run: echo "/root/ycsb/bin/ycsb run mongodb -s -P /root/ycsb/workloads/workloada -p mongodb.url=mongodb://127.0.0.1:27017 -p
workload=com.yahoo.ycsb.workloads.CoreWorkload -p updateretrycount=1000 -p mongodb.writeConcern=normal -p mongodb.database=ycsb -p
recordcount=5000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=32 -p insertretrycount=10 -p
readretrycount=1000 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p operationcount=1800000 -p fieldnameprefix=f -p maxexecutiontime=180 -p
mongodb.readPreference=primaryPreferred -p measurementtype=timeseries -p reconnectiontime=1000 -p fieldlength=10 >
/root/ycsb/2014-07-15_02-13_mongodb_workloada.out 2> /root/ycsb/2014-07-15_02-13_mongodb_workloada.err" | at 02:13 today
[10.0.32.38] out: job 23 at 2014-07-15 02:13
[10.0.32.38] out:
Done.
Disconnecting from 10.0.32.38... done.
使用 merge.py 得到具体数值
[root@mysqlstd ycsb]# ./bin/merge.py
OVERALL OVERALL READ READ READ READ READ READ READ READ READ UPDATE UPDATE UPDATE UPDATE UPDATE
UPDATE UPDATE UPDATE UPDATE CLEANUP CLEANUP CLEANUCLEANUP CLEANUP CLEANUP CLEANUP CLEANUP CLEANUP
RunTime Throughput Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency

10. www.vmcd.org
99thPercentileLatency Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency
99thPercentileLatency Operations Retries Return=0 Return=[^0].* AverageLatency MinLatency MaxLatency 95thPercentileLatency
99thPercentileLatency
1 61156 28.58264111 1665 747000 918 747 927.7820691 0.132 6630.776 83 33000 50 33 2075.808675 0.504 9767.828
32 0 18.08734375 0.465 207.159
Total 61156 28.58264111 1665 747000 918 747 927.7820691 0.132 6630.776 83 33000 50 33 2075.808675 0.504
9767.828 32 0 0 0 18.08734375 0.465 207.159
[mongo@mysqlstd ~]$ /data/mongodb/mongodb/bin/mongostat
connected to: 127.0.0.1
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn
time
*0 5568 5369 *0 0 5381|0 0 3.06g 6.47g 282m 9 admin:0.9% 0 0|0 0|0 1m 894k
95 02:14:05
*0 4298 6267 *0 0 6279|0 0 3.06g 6.47g 282m 6 admin:0.6% 0 0|0 1|0 1m 962k
96 02:14:06
*0 4675 6119 *0 0 6066|0 0 3.06g 6.47g 282m 2 admin:0.0% 0 95|0 1|0 1m 948k
92 02:14:07
*0 4137 4866 *0 0 4948|0 0 3.06g 6.47g 282m 18 admin:2.1% 0 0|0 0|0 1m 790k
91 02:14:08
*0 4568 5904 *0 0 5922|0 0 3.06g 6.47g 282m 4 admin:0.1% 0 0|0 0|0 1m 927k
92 02:14:09
*0 4727 6034 *0 0 6046|0 0 3.06g 6.47g 282m 5 admin:0.0% 0 0|0 0|0 1m 949k
90 02:14:10
*0 4991 5673 *0 0 5690|0 0 3.06g 6.47g 282m 3 admin:0.9% 0 0|0 0|0 1m 914k
94 02:14:11

11. www.vmcd.org
*0 4740 5173 *0 0 5183|0 1 3.06g 6.47g 282m 7 admin:0.1% 0 0|0 0|0 1m 839k
94 02:14:12
*0 4332 5493 *0 0 5510|0 0 3.06g 6.47g 282m 8 admin:0.9% 0 0|0 0|0 1m 866k
94 02:14:13
*0 4980 5583 *0 0 5592|0 0 3.06g 6.47g 282m 8 admin:0.0% 0 0|0 0|0 1m 901k
97 02:14:14
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn
time
*0 5750 5030 *0 0 4997|0 0 3.06g 6.47g 282m 20 admin:1.8% 0 94|0 1|1 1m 853k
97 02:14:15
*0 4884 5509 *0 0 5578|0 0 3.06g 6.47g 282m 10 admin:0.1% 0 0|0 0|0 1m 894k
97 02:14:16
*0 5733 5773 *0 0 5784|0 0 3.06g 6.47g 282m 5 admin:0.0% 0 0|0 0|0 1m 952k
92 02:14:17
*0 5178 5202 *0 0 5219|0 0 3.06g 6.47g 282m 14 admin:0.0% 0 0|0 0|0 1m 861k
95 02:14:18
*0 4179 5680 *0 0 5688|0 0 3.06g 6.47g 282m 8 admin:0.0% 0 0|0 0|1 1m 884k
93 02:14:19
*0 4879 5695 *0 0 5707|0 0 3.06g 6.47g 282m 11 admin:0.1% 0 0|0 0|0 1m 911k
93 02:14:20
*0 5271 5402 *0 0 5413|0 0 3.06g 6.47g 282m 12 admin:0.0% 0 0|0 0|0 1m 887k
95 02:14:21
*0 4583 4852 *0 0 4867|0 1 3.06g 6.47g 282m 11 admin:0.0% 0 0|0 0|0 1m 795k
93 02:14:22
*0 6654 4956 *0 0 4967|0 0 3.06g 6.47g 282m 10 admin:1.5% 0 0|0 0|0 1m 881k
95 02:14:23

12. www.vmcd.org
REF:
http://www.aerospike.com/blog/entrepreneurs-break-all-the-rules-aerospike-goes-open-source/
https://github.com/thumbtack-technology/ycsb
http://www.aerospike.com/wp-content/uploads/2013/02/Ultra-High-Performance-NoSQL-Benchmarking_zh-CN.pdf
https://github.com/brianfrankcooper/YCSB/wiki
http://labs.yahoo.com/news/yahoo-cloud-serving-benchmark/