Level: Introductory

Harini Srinivasan (harini@us.ibm.com), Software Engineer, IBM　
James Conallen (jconallen@us.ibm.com), Senior Solution Engineer, IBM　
Eoin Lane (eoinlane@us.ibm.com), Senior Solution Engineer, IBM　

24 Oct 2005

The requester side caching pattern mediates the interaction between one or more clients and one or more data providers. The mediation can make access to the data faster and less expensive. This general pattern offers variations to meet different design goals and issues. The pattern helps developers make and document design decisions around the cache and policies.

Overview

The vast majority of inter-application, Web, and Web services requests involve information retrieval as opposed to information updates. For example, in the area of presentation-layer-to- business-logic-layer communications, 60 information requests for one update is common in on-line shopping. For some applications, such as stock trading, the ratio can be well over 100 to one. Even in applications such as customer self-service we commonly find ratios in excess of 10 to one. These information requests often play a major role in overall application performance. The information being requested tends to change much less frequently than it is requested.

Often when the read-to-update ratio is high, the data is not very volatile; that is, the changes occur relatively slowly. In other words, the application use cases follow a "read-mostly" scenario.

Back to top

Problem

Requesters in a request/response scenario often experience a perceived performance problem. An example here would be a customer retrieval system where larger blocks of customer information need to be retrieved repeatedly. These performance problems can be due to a number of issues:

• Network latency -- the amount of time the requested information is on the wire. This can be particularly true of Web services requested where the verbose ASCII nature of the payload can aggravate the problem.
• Immutable remote service or interface -- accessing of an immutable remote service with no server side caching capability enabled on that service or implementation.

Back to top

Forces

The forces on a requester in a typical request response scenario are as follows:

• The network latency associated with request response invocation, especially in a Web service request where the verbose ASCII nature of the payload aggravates the problem.

• Processing of large amount of data on the requester side will impact performance if this data has to be fresh from the provider every time.

• A perceived performance may also be required on the requester side if the provider implementation is unchangeable.

Pattern specifications and implementations When we talk about Patterns (and let's use the Gang of Four "Design Patterns" as an example) people are interested in two different elements: One is the actual text that describes the pattern. This is what you would find in the book, say in the chapter on the Adapter pattern. Then there is the design tooling elements that implement that pattern. For instance, there are UML artifacts in Rational® Software Architect that implement the different design patterns from the GOF book. So, if you wanted to apply an Adapter to one of your designs, you'd pull that element off of the palette and make some manipulations in the tool to actually transform your design to use the pattern. The first thing we've called a "Pattern Specification". The second thing we've called a "Pattern Implementation".

Solution

The requestor side cache pattern solves this performance problem by providing a cache-aware proxy of the service on the requester side. Typically it "wraps" the target component's interface with a caching component co-resident with the requestor that provides the same interface as the target component. However, the caching component remembers the results of information requests made to the underlying target component so that if the same information is requested again it can be supplied from the caching component's memory rather than requesting it from the target component. At no point is the requester component aware that a cache is present as he is not exposed to any caching APIs. This is illustrated in Figure 1.

The caching component presents the same interface as the target component so the impact on the requesting application design is minimal. From the requester's point of view the target component just got much faster. From the target component's point of view the number of requests it is receiving from the requestor just went way down. Thus, caching to accelerate requests to a component has minimal impact on application logic and is quite simple -- actually transparent -- to use.

Class Diagram

The class diagram shows the decorator (Ref GOF Decorator) nature of the pattern. The provider is the ServiceImpl class which implements the IService interface. This interface typically has operations like getItem() and getItems() where the Item is some entity. The getItem() operation takes a primary key to identify the Item whereas the getItemKeys() typically takes a selection criteria that can be converted to a set of primary keys by the getItemKeys() operation. The IService interface may also have a changeItem() operation that by definition causes a change to the internal structure of Item (this is included to handle staleness in this case). The decorator is then the CacheServiceImpl class. The CacheServiceImpl implements the IService interface and then wrappers the ServiceImpl by providing caching capabilities to the getItem() and getItems() operations.

Sequence diagram

The Sequence diagram shows the requester using the CacheServiceImpl client side proxy to call getItem() The getItem() implementation will first check the cache to see if the Item can be found there. Failing to find the Item in the cache the getItem() will then get the Item from the provider service. The Item will then be stored in the cache for future reuse. The getItems() method just uses the getItemKeys() operation to get a unique set of primary keys and then calls the getItem() operation.

Back to top

Participants

Applying the pattern

This pattern should be used to speed up and reduce the costs associated with accessing information provided by another component. The user should understand the limits and requirements associated with caching as discussed above to ensure that the use of this pattern will not compromise the function of the application and will be effective at accelerating access to data.

The caching pattern makes some strong assumptions about the structure of the target component interface. For example, it requires that keys be single, explicit parameters and it requires that the data items to be cached also be explicit parameters or members of a list. If these restrictions are not met by a particular target component interface then it may be necessary to wrap the target component with a mapping class that has an interface that meets the pattern requirements.

The caching pattern provides caching support for two types of operations on the providing component:

1. Those that take a single data item key and return a single data item
2. Those that takes criteria and return a list of data items. In this case the providing component also must have an operation that takes the same criteria and returns a list of keys.

Parameters

• Service: The interface/class that contains the operation that we want to accelerate via caching

• getItems: The operation on the service interface/class that is used to get items given a set of criteria

• getItemKeys: The operation on the service interface/class that is used to get keys given a set of criteria

• getItem: The operation on the service interface/class that is used to get a single item given an item key

• changeItemKey: The parameter in the change item operation that corresponds to the key of the item

• Cache size: The size of the cache

• Clustering: A Boolean value where true implies the underlying topology is clustered and false implies the underlying topology is not clustered

• Timeout: The time out value (measured in milliseconds) after which an item has to be evicted from cache

Implementation

The implementation will provide a proxy that is cache-aware instead of the class that was used to access the providing component. The proxy has the same interface so no changes should be needed in code that used the provided component. However, the code that is accessing the providing component will need to be modified to use the newly generated service class to access the service operations.

In addition, it may be necessary to add logic to the requesting component to remove data items from the cache that have changed in the providing component.

Consequences

There are certainly consequences involved with these actions,. You can expect any of the following:

• Access to data provided by another component is speeded up.
• Cost of access to data provided by another component is reduced.
• Caching retains data in the requesting component's memory and therefore increases the component's memory footprint. The cache capacity must be set carefully to limit this increase to the amount that is most effective. This can be difficult in a clustered environment.
• If the underlying data associated with a key changes it is necessary to remove any data cached under that key from the cache. Two basic strategies for doing this are discussed above. The user of this pattern must evaluate the options and make good choices, or the requesting components function can be compromised.

Back to top

Considerations in using caching

There are also a number of considerations that must be factored in to the use of caching.

Data Keys

A basic requirement for caching is that the data to be cached can be identified explicitly and uniquely. The identifier for a cacheable data item is called its key. Fundamentally, the cache supports operations to insert a data item with a specified key and then retrieve it later with the same key. So it must be true that if two data items represent different information they must have different keys, it is also necessary that each data item must have only one key.

Data Volatility

If the data associated with a key changes after a previous version has been placed in the cache then retrieving data with this key from the cache will retrieve obsolete data. This problem of data volatility is usually the biggest limitation to using a cache. There are two basic strategies for handling it, time out of items in the cache and an explicit invalidation of items in the cache:

• Time out: Data is frequently subject to change, but it is ok to retrieve out-of-date data for a short period. For example, product prices in an ecommerce site change slowly and it is usually fine to give out an old price for some number of minutes after the price has changed. When this is the case, the caches supported in this pattern allow a time-out value to be set. Whenever a data item retrieval is attempted, a check is made to see how long ago the data item was put in the cache. If the time is too great then the data item will be removed from the cache and the cache will behave as if the item was not there in the first place.
• Explicit invalidation: It is possible that all sources of change to data items held in the cache can cause changed data items to be removed from the cache immediately. The caches supported by this pattern allow for items to be explicitly removed from the cache to support this. However, it is the responsibility of the user of this pattern to ensure that the removals occur. This pattern supports one particular pattern of change. Namely it allows operations in the service's interface that result in changes to items to be wrapped with logic that will remove the changed items from the cache if the changed item's keys can be identified explicitly in the service interface.

Working set size

A cache is said to have a working set size. This is the number of items that need to be kept in the cache so that most requests for an item will be satisfied from the cache. Generally, and specifically with the cache implementations supported by this pattern, a cache will operate with a bounded number of items that it can contain, referred to as its capacity. If more items than this are added then some items will be evicted. The eviction policy used in both of the cache implementations supported by this pattern is least-recently-used or LRU. This policy evicts the items that have gone the longest without being referenced. Under this policy if the cache's capacity is large enough then its working set will be maintained in the cache and it will be effective at accelerating requests to the target component. However if it is too large then space will be wasted. If it is too small the working set will not fit and the cache will be much less effective.

Working set size can be difficult to specify, since it varies widely based on reference patterns which may vary from application to application and may vary even during the execution of a single application.

The cache supported by this pattern can be used in a clustered environment which requires that its capacity be set at deployment time. You will need to estimate the working set for the cache in your particular application to set this well.

The cache supported by this pattern that runs as part of a single application instance only requires that an initial capacity be provided and then it will adjust it capacity based on runtime measurements to attempt to always contain its working set, but with very little left over capacity.

Back to top

Related patterns

The following are related patterns are related to the requester side caching pattern

• GOF decorator pattern:This requester sIde caching pattern is essentially the GOF decorator pattern as outlined by the Gamma et al in their Book called Design Pattern. The decorator patterns is used to provide an alternative implementation of an interface where this alternative implementation wraps or decorates the previous implementation.
• GOF facade pattern: This facade pattern can be used to convert the provider's component interface to a form that is compatible with the interface required by the Requester Side Caching pattern.

Back to top

Known usages

This pattern is used by the WebSphere® Application Server client-side Web services cache (which uses Dynacache) as the supported caching mechanism. Since the requester side caching capability is built into the web service client proxy, on a WebSphere platform, this caching pattern becomes redundant. However, outside WebSphere environments, and in situations where a custom in-memory cache may be preferable to Dynacache, this pattern solution can be effectively leveraged.

Back to top

Conclusion

We have examined the requester side caching pattern specification in detail here. This pattern specification follows the outline of patterns as specficied in the book titled "Design Patterns" by Gamma et al. Part 2 in this series returns to this pattern specification and provides an implementation of this pattern specification using the model driven development environment of IBM flagship development product, Rational® Software Architect.

Resources

• Visit IBM's Pattern Solutions to find out what IBM is doing around patterns and reusable assets.
  
  
• Learn more about the OMG standard for Reusable Asset Specification, version 2.2 by reading the specification
  
  
• Visit the IBM developerWorks SOA and Web services zone to expand your skills.
  
  
• Vist the Enterprise Integration Solutions (EIS) site to learn more about what IBM is doing around SOA. Enterprise Integration Solutions (EIS) of IBM Software Group is part of IBM's corporate Software Strategy and Technology organization. The team controls and develops all of IBM's SOA related software products.
  
  
• Stay current with developerWorks technical events and Webcasts.
  
  
• Get an RSS feed for this series. (Find out more about RSS.)

• Download a free trial version of Rational RequisitePro Rational RequisitePro V2003.06.15.
  
  
• Download a free trial version of Rational Software Architect Rational Software Architect V6.0.
  
  
• Build your next development project with IBM trial software, available for download directly from developerWorks.
  
  
• IBM Alphawork provides a reference implementation of the RAS called Reusable Asset Specification Repository for Workgroups, this reference implementatiion is available for download directly from alphaWorks.
  
  
• Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  

• Participate in the discussion forum.
  
  
• Participate in developerWorks blogs and get involved in the developerWorks community.
  
  

About the authors

		Harini Srinivasan is in IBM Enterprise Integration Solutions. Her research and technology experiences include compiler and run-time analysis of data parallel and control parallel applications for massively parallel computers; compiler and runtime optimizations of object-oriented programs, JVM runtimes and optimization, and performance analysis, program understanding and debugging tools for enterprise real-world applications. More recently, she has been working in the area of model-driven approach to building service-oriented applications. She has been an employee of IBM for the last 11 years, the first 10 years of which as a Research Staff Member at the IBM T.J. Watson Research Center.

		Jim Conallen is a software engineer in IBM Rational's Model Driven Development Strategy team, where he is actively involved in applying the Object Management Group's (OMG) Model Driven Architecture (MDA) initiative to IBM Rational's model tooling. Jim is also active in the area of asset-based development and the Reusable Asset Specification (RAS).

		Dr. Eoin Lane, Senior Solution Engineer, is the lead for harvesting and developing of application pattern from key IBM SOA engagements and driving those patterns through IBM pattern governance process to accelerate adoption. Eoin also specializes in Model Driven Development (MDD), asset based development and Reusable Asset Specification (RAS) to facilitate SOA development.

Original link: http://www.ibm.com/developerwork...