Presentation I did on AppFabric velocity while it was in beta, as in before it was called AppFabric

1. Dennis van der Stelt
Velocity, distributed caching
Session Code: NE.13

2. Introduction
• Class-A
–
–
–
–
Kennisprovider
Microsoft development
Training / Coaching
www.class-a.nl
• Dennis van der Stelt
– Trainer / Coach
– Weblog at http://bloggingabout.net/blogs/dennis/

3. Agenda
•
•
•
•
•
•
Challenges for webfarms
What is Velocity?
Options in caching in Velocity
Velocity features
Caching strategies
Best practices

4. Challenges for webfarms
Web server A
Load balancer
Web server B
Web server C

5. Solution : sticky sessions
Web server A
Load balancer
1.
2.
3.
4.
Web server B
Uneven load balancing
Lost sessions
Single point of failure
Need to drain state
Web server C

6. Solution : Use DBMS
Web server A
Load balancer
Web server B
1. Doesn’t scale
2. Creates a performance bottleneck
3. Clustering is costly and fault
sensitive Web server C

7. What is Velocity?

8. What is Velocity?
• An explicit, distributed, in-memory application cache for
all kinds of data (CLR objects, XML, Binary data, etc.)
– Flows "memory" across machines into a unified cache
Clients can be spread
across machines or
processes
Unified Cache View
Clients Access the
Cache as if it was a
large single cache
Cache Layer
distributes data
across the various
cache nodes

9. Velocity in a webfarm
Web server A
Load balancer
Web server B
Web server C

10. Why Velocity?
* Share data across applications
No more sticky routing
* Peformance
Operation
Read 2k
MSDN Forums
Throughput
Latency
30,000 / sec
3 – 4 ms
Write 2k
18,000 / sec
5 Velocity Servers, 40GB Cache
98% of all calls to database come from cache
Result is from 60% load on SQL Server, down to 2%
* Scale out
3 ms
Operation
Throughput
Read 2k
1
30,000 / sec
Read 2k
2
58, 600 / sec
Read 2k
Velocity
Servers
3
85, 500 / sec

11. Key features of Velocity
• Caches any serializable CLR object
• Enterprise scale
– Up to hundreds of computers
– Dynamic scaling
• Runs as a service
• Automatic load balancing

12. Key features of Velocity
• Tight integration with ASP.NET
– Use as session store
– Cache application data
• Integration with administration and
monitoring tools as System Center, etc.
• Cache-aside architecture for version 1
• Support for multiple client languages
– PHP, C#, C++, etc.

13. Velocity installation
• Needed for installation
– .NET Framework 3.5
– Windows XP/Vista/Server (2003/2008)
• Firewall exceptions on port 22233-22235
• Configuration
– SQL Server
– Network share
single-point-of-failure?
10 concurrent connections?

14. Basic terminology in Velocity
•
•
•
•
•
•
•
Cache host
Cache cluster
Cluster configuration storage
Machine 1
Machine
Machine
Named cache
2Cache
2Cache
Cache
Cache
host A
host B
host C
host D
Region
Named cache : Product catalog
Named cache : ShoppingCart
Cache item
Tags
Region A

15. Working with Velocity
// Create instance of CacheFactory, which reads app.config
DataCacheFactory factory = new DataCacheFactory();
// Get a named cache from the factory
DataCache cache = factory.GetCache("default");
// Cache.Put(string key, object value)
cache.Add("SDC", new SDNConference());
// Cache.Get(string key);
var meeting = (SDNConference)cache.Get("SDC");
// Via indexers is also an option
cache["what"] = new Object();
Object o = cache["what"];

16. Types of cache
Partitioned
• Data partitioned across all nodes
• Used for scale & availability
Replicated
• Data replicated across all nodes
• Used for high availability
• All reads occur on primary host
Local
• Payload stays in object form
• No serialization
• No extra network hops

17. Partitioned cache
Application 2
Application 1
Put(“T5”, “D”)
Routing table
Cache1
Routing table
Primary Regions
T1, “A”
Routing table
Cache2
Routing table
Primary Regions
Cache3
Routing table
Primary Regions
T5, “D”
T3, “A”

18. Replicated cache (high availability)
Application 2
Application 1
Put(“T5”, “D”)
Get(“T5”)
Routing table
Cache1
Routing table
Primary Regions
T1, “A”
Routing table
Cache2
Routing table
Primary Regions
Cache3
Routing table
Primary Regions
T5, “D”
T3, “A”
Secondary Regions
T5, “D”
Secondary Regions
T3, “A”
Secondary Regions
T1, “A”

19. Cache clients
Simple client
Routing Client
• Has to request
where objects live
• Has knowledge of
where objects live
• Keeps in sync with
hosts
• Performance
benefits
• Extra network
communication
Local cache
• No pre-installed
host needed
• Your application is
the host
• Configurable
• Stored in
deserialized state!

20. Partitioned cache fronted by local cache
Put(T5, “D”)
Get(T5)
Velocity Client2
Velocity Client1
Local cache
Local cache
T5, “D”
Routing layer
T5, “D”
Routing layer
Cache1
Cache2
Cache3
Primary Regions
Primary Regions
Primary Regions
T1, “A”
T5, “D”
T3, “A”

21. Velocity features

22. Regions
// Without regions
cache.Put("MyKey", sdnConference);
// With regions
cache.CreateRegion("MyRegion", true);
cache.Put("MyKey", sdnConference, "MyRegion");
var result = (SDNConference)cache.Get("MyKey", "MyRegion");
• Regions don’t distribute
• But will replicate when high availability is on!

23. Tags
var starWarsTag = new DataCacheTag("StarWars");
var tags = new List<DataCacheTag>();
tags.Add(starWarsTag);
tags.Add(new DataCacheTag("Force"));
tags.Add(new DataCacheTag("Sith"));
cache.Add("MyKey", "A New Hope", tags, "StarWarsRegion");
var result =
cache.GetObjectsByTag(starWarsTag, "StarWarsRegion");
foreach (var item in result)
{
Console.WriteLine("{0} has value of {1}",
item.Key, item.Value);
}

24. Optimistic locking
DataCacheItemVersion versionWillChange;
DataCacheItemVersion versionWithError;
// First get the current version 2 times
cache.Get("MyKey", out versionWillChange);
cache.Get("MyKey", out versionWithError);
// We change the key, version hasn't changed in Velocity yet.
cache.Put("MyKey", "MyNewValue", versionWillChange);
// Version has changed with previous update, this will #fail
cache.Put("MyKey", "MyErrorValue", versionWithError);

25. Pessimistic locking
DataCacheLockHandle lockHandle = null;
DataCacheLockHandle secondLockHandle = null;
// Lock our object
cache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out
lockHandle);
// This will still work
string result = (string)cache.Get("MyKey");
// Try to lock for 2nd time -> #fail
cache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out
secondLockHandle);
// This will break the lock!!!
cache.Put("MyKey", "MyNewValue");

26. Notification callback
DataCacheOperation filter = DataCacheOperation.AddItem;
cache.AddItemLevelCallback("MyKey", filter, callback);
cache.AddRegionLevelCallback("Region", filter, callback);
cache.AddFailureNotificationCallback(failCallback);
cache.Add("MyKey", "MyInitialValue");
public static void callback (string myCacheName,
string myRegion, string myKey,
DataCacheItemVersion itemVersion,
DataCacheOperation OperationId,
DataCacheNotificationDescriptor nd)
{
//display some of the delegate parameters
Console.WriteLine("Region : " + myRegion);
Console.WriteLine("Key
: " + myKey);
}

27. Cluster configuration
<dataCacheClient deployment="routing">
<localCache
isEnabled="true"
sync="TTLBased"
objectCount="100000"
ttlValue="300" />
<clientNotification pollInterval="300" />
<!-- cache host(s) -->
<hosts>
<host
name="localhost"
cachePort="22233"
cacheHostName="DistributedCacheService"/>
</hosts>
</dataCacheClient>

28. ASP.NET Session integration
• SessionStoreProvider class
– Plugs into ASP.NET Session store
– Stores session state in Velocity
• Scale
– Session information available at all ASP.NET
Nodes
• High availability
– Session data is backed up on addditional
machines
– Resilient to machine or process failures

29. Performance

30. Achieving scale and performance
• Scale on data size
– More machines => more memory to cache
• Scale on cache throughput
– Throughput : Operations/sec from entire cache
– More machines : keys distributed across more
machines => better throughput
• Performance
– High performance by scaling out data and processing
– Increased memory
– Increased processing
• Workload distributed across multiple cache nodes

31. Achieving scale and performance
Server 2 Added
Throughput Increases
Latency Decreases
Until Server Saturation
Server 3 Added
Throughput Increases
Latency Decreases
Load
Single Server
Throughput Increases
with Increasing Load
Until Server Saturation
Throughput
Latency

32. Caching strategies

33. Velocity
• Is Velocity next-next-finish?
– Think about the consequences!
– From in-proc. cache to out-of-process cache
– Multiple network cards for higher bandwith
• Don’t disturb your normal web visitors
• Velocity can bring network card to its knees
– Computers are communicating data
• Can you transfer lots of data, allowed to do so?
• What about security?

34. Best practices
• Develop a caching strategy
– When and what to cache
– Selection of cache keys
• Create guidelines for caching
– Make sure developers live by the strategy
– Caching is hard to do right
• Code once, measure twice

35. Best practices
• Cache close to where you need the items
– Usually presentation or service layer
• Cache wisely
– Think about cache item lifetimes, eviction
– Don’t cache to get performance, but to
improve performance
• Define a key naming convention
– Avoid conflicts of cache entries

36. Velocity and the future
• Available mid 2009 (huh?)
• Velocity is cache provider in .NET 4.0
• Version 2
– Will be available in the cloud
– LINQ over Velocity
– Output caching provider
– Cache through
• Read through & Write behind
– Grid computing

37. Review
• Distributed cache is the future for
performance demanding enterprises
– Linkedin, Slashdot, Facebook, Flickr,
Wikipedia
• Velocity makes this possible
• A giant hashtable with performance,
scalability and failover.

38. Thank you for your attention
• Dennis van der Stelt
– Dennis@BloggingAbout.NET
– http://twitter.com/dvdstelt/
– http://bloggingabout.net/blogs/dennis/
Resources to more information can be found here, incl. this slidedeck.

39. Evaluation form
Vul je evaluatieformulier in en maak kans
op een van de prachtige prijzen!!
Fill out your evaluation form and win one of
the great prizes!!
Session Code: NE.13