2. Caching – The Need
Web applications are typically accessed by many
concurrent users.
Usually, the application's data is stored in a
relational database or file system, and it takes
time and costs overhead to access these data
sources.
Database-access bottlenecks can slow down or
even crash the application if it receives too many
simultaneous requests.
Object caching is one technique that overcomes
this problem.
Caching is one of most important factors if we
think about application performance. Most
applications need a caching implementation to
improve performance.
3. Advantages
Caching is meant for High scalability and
performance
It reduces number of trips to the database
or other data sources, such as XML
databases or ERP (enterprise resource
planning) legacy systems
It avoids the cost of repeatedly recreating
objects
It shares objects between threads in a
process and between processes
It efficiently uses process resources
4. Disadvantages
Memory Size: The cache may consume significant
heap space in the application server. JVM
memory size can become unacceptably huge if a
lot of unused data is in the cache and not
released from memory at regular intervals.
Synchronization complexity: Depending on the
kind of data, complexity increases because
consistency between the cached data's state and
the data source's original data must be ensured.
Otherwise, the cached data can fall out of sync
with the actual data, which leads to data
inaccuracies.
Durability: Cache invalidation/deletion on server
crash
5. Caching by HttpSession
The HttpSession object (in the servlet
package) also allows objects to be cached,
but lacks the concepts of sharing,
invalidation, per-object expiration,
automatic loading, or spooling, which are
the essential elements of a caching
framework.
Also HttpSession object becomes quite
heavy, and are not meant to store huge
amount of data.
6. Caching by EJB
EJB provide a way to store an object
in memory and perform the object
lookup based on a key.
But none of these methods provide
any mechanism for either removing
the object from memory when it's no
longer needed or automatically
creating the object when it's
accessed after expiration
7. Caching Algorithms
Cache should be invalidated with
some algorithm if not needed as they
consume memory and got limited
capacity.
Few Examples: least frequently used
(LFU), least recently used (LRU),
most recently used (MRU), first in
first out (FIFO), last access time, and
object size.
8. FIFO (First In First Out)
Items are added to the cache as they are
accessed, putting them in a queue or buffer and
not changing their location in the buffer; when
the cache is full, items are ejected in the order
they were added. Cache access overhead is
constant time regardless of the size of the cache.
The advantage of this algorithm is that it’s simple
and fast; it can be implemented using just an
array and an index. The disadvantage is that it’s
not very smart; it doesn’t make any effort to
keep more commonly used items in cache.
Summary for FIFO: fast, not adaptive, not scan
resistant.
9. LRU (Least Recently Used)
Items are added to the cache as they are accessed; when
the cache is full, the least recently used item is ejected.
Cache access overhead is against constant time. This
algorithm is simple and fast, and it has a significant
advantage over FIFO in being able to adapt somewhat to
the data access pattern; frequently used items are less
likely to be ejected from the cache. The main disadvantage
is that it can still get filled up with items that are unlikely to
be reaccessed soon; in particular, it can become useless in
the face of scans over a larger number of items than fit in
the cache. Nonetheless, this is by far the most frequently
used caching algorithm [4].
Summary for LRU: fast, adaptive, not scan resistant
10. LFU (Least Frequently Used)
Frequency of use data is kept on all items. The
most frequently used items are kept in the cache.
The advantage is that long term usage patterns
are captured well, incidentally making the
algorithm scan resistant as well; the
disadvantage, besides the larger access
overhead, is that the algorithm doesn’t adapt
quickly to changing usage patterns, and in
particular doesn’t help with temporally clustered
accesses.
Summary for LFU: not fast, captures frequency of
use, scan resistant
11. Simple time-based expiration
Data in the cache is invalidated
based on absolute time periods.
Items are added to the cache, and
remains in the cache for a specific
amount of time.
Summary for Simple time-based
expiration: Fast, not adaptive, not
scan resistant.
12. Extended time-based expiration
Data in the cache is invalidated based on
relative time periods. Items are added to
the cache, and remains in the cache until
they are invalidated at certain points in
time, such as every five minutes, each day
at 12:00AM etc.
Summary for Extended time-based
expiration: Fast, not adaptive, not scan
resistant.
13. TTL (Time to live) expiration
Data in the cache is invalidated by
specifying the amount of time the
item is allowed to be idle in the
cache after last access time [5].
17. Main elements of an object-
caching framework
A typical caching framework contains
components such as
a CacheObject, CacheObjectKey, Cac
he, CacheManager, and a
CacheLoader.
18. ICacheManager
ICacheManager is the main interface (contract)
that a client program uses to handle all the
operations related to caching (i.e., storing,
accessing, and releasing the data in the cache).
The client program can be a JSP (JavaServer
Pages) page, a Struts action class or a POJO
(plain old Java object). This interface was created
to hide all the caching implementation details
from the client so if we needed to switch to a
different third-party caching API in the future, we
wouldn't need to change any of the client code.
19. BaseCacheManager
BaseCacheManager is the main class in the Web portal
caching framework. It's the base implementation
of ICacheManager. This class was created to centralize all
the cache-related methods in one class. It's designed as a
singleton to ensure one and only one instance
ofICacheManager is created in the servlet container's JVM.
In a clustered environment where multiple Web
server/servlet container instances accept Web requests, a
separate ICacheManager instance will be created in each
JVM. If we switch to a different caching API later, this is the
only class that must be modified to work with the new
cache API. Also, if we switch to a JCache-compliant caching
implementation, the cache manager should require minimal
changes.
20. ICacheLoader
The ICacheLoader interface implements the actual data-access logic in the
Web client. All client programs that need to use the caching mechanism
must implement this interface. It has one method
calledloadCacheObject() and takes two input parameters, a string to specify
the cache region name and an object to specify the cache key. This way,
the cache manager knows which client program to use (to
execute loadCacheObject()) to reload the object in the cache when the
cached data expires after the specified time-to-live has elapsed.
It is good practice for the caching service to load objects automatically as
needed rather than using the application to directly manage objects that
use the cache. When an application directly manages objects, it uses
the CacheAccess.put() method to insert objects into the cache. To take
advantage of automatic loading, we instead use a CacheLoader object and
implement its load method to put objects into the cache.
Note that the caching framework does not handle the creation of objects
that need to be cached in a Web application, i.e., the data-access logic that
retrieves the data from the data source is not coded in the caching classes.
It relies on the client program to define the actual data-access logic.
Technologies like Java Data Objects (JDO) are typically used to encapsulate
the data-access logic in an enterprise Web application.
21. ICacheKey
The ICacheKey interface was created to hide the
specific logic used to create a cache key.
Sometimes the cache key may not be a simple
string. It may be as complex as the combination
of multiple objects, and getting these values from
the data source involves not one, but several,
lookup methods. In this case, ICacheKey can
define all the complex logic involved in creating
the cache key. This way, the cache-key creation
logic is defined in a separate class. I wrote a
sample class called TestCacheKey that
implements this interface and overrides
the getCacheKey() method to illustrate how to
use this interface.
22. CacheRegion
A CacheRegion is defined as an
organizational namespace for holding a
collection of cache objects. Objects with
similar characteristics (such as time-to-live
and business use) should be cached in the
same cache region so they can all be
invalidated simultaneously if needed. To
eliminate any synchronization issues that
could cause poor performance, I used a
separate instance ofCache for each cache
region.
23. Cache Loader
In conjunction with eviction policies, a
cache loader allows a user to maintain a
bounded cache for a large backend
datastore. Frequently used data is fetched
from the datastore into the cache, and the
least used data is evicted, in order to
provide fast access to frequently accessed
data. With a cache loader, the Java object
cache automatically determines if an
object needs to be loaded into the cache
when the object is requested.
24. Locking
Two kinds of locking are normally adopted for
Java cache framework: pessimistic locking and
optimistically locking.
Pessimistic locking is an approach where an entity
is locked for the entire time during a transaction.
A lock either limits or prevents other users from
working with the entity.
Optimistic lock, which involved versioning data
and maintaining copies for each transaction,
validating copies upon transaction commit with
the cached entity. This approach led to a very
highly concurrent setup for a read-heavy system
where readers are never blocked by concurrent
writers, and also overcame the potential for
deadlocks which may occur in pessimistically
locked cache framework.
25. Different Caching Technologies
There are several:
Terracotta (open source, based on Mozilla Public License);
Oracle Coherence (formerly Tangosol Coherence;
commercial; based on JSR 107, which was never adopted
officially);
GigaSpaces (commercial; based on JavaSpaces API, part
of Jini);
GridGain, which you mentioned (open source: LGPL);
Memcached with a Java client library (open source:
BSD License;
EHCache (open source: Apache Software License;
OSCache (open source: modified Apache License; and
26. Grid Computing
Grid Computing is a way to distribute
your caching and computations
across multiple computers (nodes).
They are split them into 2
categories: Compute Grids and Data
Grids.
27. Compute Grid
Compute Grids allow you to take a computation, optionally
split it into multiple parts, and execute them on different
grid nodes in parallel. The obvious benefit here is that your
computation will perform faster as it now can use resources
from all grid nodes in parallel. One of the most common
design patterns for parallel execution is MapReduce.
However, Compute Grids are useful even if you don't need
to split your computation - they help you improve overall
scalability and fault-tolerance of your system by offloading
your computations onto most available nodes.
Some Compute Grid vendors:
- GridGain - Professional Open Source
- JPPF - Open Source
28. Data Grid
Data Grids allow you to distribute your data
across the grid. Most of us are used to the term
Distributed Cache rather than Data Grid (data
grid does sound more savvy though). The main
goal of Data Grid is to provide as much data as
possible from memory on every grid node and to
ensure data coherency.
Some Data Grid/Cache vendors:
- Oracle Coherence - Commercial
- GemStone - Commercial
- GigaSpaces - Commercial
- JBossCache - Professional Open Source
- EhCache - Open Source
29. Writing a Price to GigaSpaces
In “classic” JavaSpaces every object in the
space needsto implement “Entry”
–GigaSpaces have optimised this out in
their most recent release
// Find the
spaceJavaSpace space = (JavaSpace)SpaceFinder.find(“jini://*/*/
mySpace”);space.write( myPrice, null, Integer.MAX_VALUE ); //
null transactions
Because Sun haven’t evolved Jini and
JavaSpaces GigaSpaces (and others) have
had to do the innovation
30. Reading a Price from GigaSpaces
// Find the
spaceJavaSpace space = (JavaSpace)SpaceFinder.find(“jini://*/*/mySpace
”);// Define and template of what we’re looking for...Price template = new
Price ( “GBP/NOK”, “SPOT”, null );Object entry = space.read( template,
null, 10000 ); // null transactionsPrice myPrice = ((Price) entry);
Optimisations and exception handling were left out for
clarity e.g.
–We would normally use a “snapshot” of the template
Notice how the query is done by an interface/template
rather than adata-centric query
–This is more in line with SOA, i.e. Service Oriented as
opposed to data-oriented
So how does GigaSpaces work?
31.
32. Writing a Price to Coherence
We don’t need to implement anything
–Since the data is distributed is must of
course be Serializable
// Create / Find a cache (using the map interface)NamedCache
map = CacheFactory.getCache(“myCache");// Write the Price to
the cache... Map.put( trade.getKey(), trade );
Creating an index...
// Create an index on the “key” map.addIndex(new
ReflectionExtractor(“getKey”, false, null);
As you can see this couldn’t be easier
33. Reading a Price from Coherence
// Create / Find a cache (using the map interface) NamedCache
map = CacheFactory.getCache(“myCache");// Retrieve the
latest spot price for GBP/NOK Price myPrice =
(Price ) map.get(“GBP/NOK-SPOT” );
At this level Coherence works in a similar
way to GigaSpaces
34. References
http://java.dzone.com/articles/compute-grids-vs-data-grids
- Compute Grids vs. Data Grids
http://jasonwordpress.wordpress.com/2010/03/27/the-
strategies-of-using-java-caching-frameworks-to-increase-
the-performance-of-j2ee-applications/ - The strategies of
using Java caching frameworks
http://www.javaworld.com/javaworld/jw-05-
2004/jw-0531-cache.html - J2EE object-caching
frameworks - JavaWorld
http://www.scribd.com/doc/193625/Caching-Grid-In-A-
Nutshell - Caching Grid in a Nutshell
