Optimization Guidance for Windows Azure Caching

• Consider Caching Options
  
  
• Set Explicit Expiration Times
  
  
• Use Local Cache
  
  
• Use Connection Pooling
  
  
• Compression and Object Size
  
  
• Locality
  
  
• Handle Retries
  
  
• Use API Improvements for Caching (Preview) on Roles
  
  

The right caching strategy can greatly enhance both throughput and latency. During the planning stages of your application development, evaluate your overall caching strategy. This strategy could include multiple storage technologies or even third-party caching providers.

For example, instead of using Windows Azure Caching to cache an object, you could choose to place that object in Windows Azure tables. This location has the advantage of being highly available due to the multiple replicas used with Windows Azure storage.

However, you might find that you require more advanced caching features, such as automatic expiration with a least-recently-used (LRU) algorithm. Or you might want the performance enhancement offered by local cache. A custom caching solution that uses a generic backend store requires you to create this code from scratch. With Windows Azure Caching, these common caching features and related APIs are already built-in. Also, performance of Caching is often superior to other backend stores, and the use of local cache accentuates this performance difference.

The following table lists some of the storage technologies that could be used to cache data and the strengths and weaknesses of each.

Sometimes it is helpful to use more than one technology to implement an overall caching strategy. For example, consider a web application for an online store. The product catalog is stored in SQL Database. To improve performance and reduce database load, the application caches the catalog and some of the most popular query results. In this example, the catalog is a large amount of data that changes infrequently. Instead of caching the catalog data with Windows Azure Caching, the catalog could be stored in a Windows Azure table. This catalog data remains in storage permanently and is only updated when the catalog changes. However, the same application chooses to store user preferences in ASP.NET session state by using the Windows Azure Caching provider. Because the catalog data does not also reside in the cache, the amount of memory required for Caching is smaller. The cache also has more room to grow in the future. The purpose of this example is not to recommend this architecture for similar applications; it shows that an overall storage strategy is more effective than putting everything in the cache without considering tradeoffs.

It is important to think carefully about expiration settings in Windows Azure Caching. Long expiration times have several potential disadvantages. First, if the cached source object changes more frequently, then the cached version could become stale. By setting the expiration time to a shorter value, you force the application to refresh the cached object at an appropriate interval. Also, long expiration times could result in filling the cache with items that are no longer being used. In the typical scenario, least-recently-used (LRU) items are evicted to make room for new cached objects. An always-full cache makes it difficult to understand the amount of memory that you actually require. It makes it harder to spot eviction patterns that point to a cache that is too small for the demand.

It is also possible to set expiration times that are too short and negate the benefits of caching. The key point is to think about the right value for expiration. Understand that the choice of expiration time might vary depending on the type of cached data.

Shared Caching

cache.Put("testkey", "testobject");

cache.Put("testkey", "testobject", new TimeSpan(0,10,0));

Caching (Preview) on Roles

Local cache is a feature of Windows Azure Caching that improves performance by reducing network requests to remote caches. Windows Azure Caching stores objects in serialized form in an in-memory cache that is distributed over multiple servers. This distributed cache is also referred to as a cache cluster. When an application requests an object from the cache, the server that stores that object is identified. That server then sends the serialized object to the requesting application over the network. The application then deserializes the object for its use. Local cache temporarily stores the object on the client machine in deserialized form. Fulfilling the request locally has two important benefits:

1. Local cache avoids the network request for the object on the cache cluster.
   
   
2. Local cache avoids the deserialization costs associated with this transfer.
   
   

These two benefits combine to reduce latency and network load.

To understand local cache, consider the following timeline where local cache is used in an application. In this example, local cache is configured with an expiration of 10 minutes:

1. The application attempts to retrieve a user profile object from the cache.
   
   
2. The cache cluster returns null, indicating that the object is not found.
   
   
3. The application retrieves the user's profile from a database and puts it into the cache.
   
   
4. Later, the application attempts to retrieve the user profile object from the cache again.
   
   
5. The cache cluster finds and returns the object to the application over the network.
   
   
6. Because local cache is enabled, the object is also stored in the memory of the virtual machine instance that requested the object.
   
   
7. After five minutes, the application on the same virtual machine instance requests the user profile object.
   
   
8. Because the object is in local cache, it is returned immediately from memory. No network call is made to the cache cluster. No processor time is spent deserializing the returned object.
   
   
9. After ten minutes, the object in local cache is invalidated. The next request for the user profile object would retrieve a fresh copy of the object from the cache cluster. This object is stored in local cache, and the process continues.
   
   

Local cache avoids repeated network requests to the cache cluster for frequently accessed objects. This feature improves performance and increases the scalability of the solution.

Before using local cache for all types of data, understand that items in local cache are not updated until they are invalidated locally. If the object changes on the cache cluster, the application would continue to use the older value until the locally stored object is invalidated. For more information, see the following sections on Shared Caching and Caching on Roles.

Local cache is enabled in the application or web configuration files in the dataCacheClient section. Multiple named dataCacheClient sections can reside in a single dataCacheClients section. This is useful in the scenario where only some types of data use local cache. You can create one configuration with local cache enable and one with it disabled. The code can create different DataCache objects that use a different configuration to cache different types of data.

Shared Caching

<dataCacheClients>
  <dataCacheClient name="default">
    <localCache isEnabled="true" sync="TimeoutBased" objectCount="100000" ttlValue="600" />
    <!-- Other cache client configuration settings here -->  
  </dataCacheClient>
</dataCacheClients>  

Caching (Preview) on Roles

<dataCacheClient name="default">
  <autoDiscover isEnabled="true" identifier="WebRole1" />
  <localCache isEnabled="true" sync="NotificationBased" objectCount="100000" ttlValue="300" />
  <clientNotification pollInterval="60" />
</dataCacheClient>

Connection pooling is a feature that reuses connections to the cache cluster for multiple DataCacheFactory objects that use the same configuration settings. After the connection pool is established, it is faster to create new DataCacheFactory objects. Those new objects can reuse a connection to the cache from the pool.

Consider the following code that places an object in the default cache.

DataCacheFactoryConfiguration config = new DataCacheFactoryConfiguration("default");
DataCacheFactory factory = new DataCacheFactory(config);
DataCache cache = factory.GetDefaultCache();
cache.Put("testkey", "testobject");

In this example, the settings for the DataCacheFactoryConfiguration object are loaded from the configuration file. The connection to the cache happens when the DataCacheFactory object is created. This operation is the most expensive in terms of time. With the latest Windows Azure SDK, connection pooling is enabled by default when you define your cache settings in the application or web configuration files. Because of this default behavior, it is important to set the size of the connection pool correctly. The connection pool size is configured with the maxConnectionsToServer attribute on the dataCacheClient element. The following example changes this value to 5.

<dataCacheClients>
  <dataCacheClient name="default" maxConnectionsToServer="5">
    <!-- Other cache client configuration settings here -->
  </dataCacheClient>
</dataCacheClients>

In this example, a maximum of five connections are used for this default configuration, regardless of the number of DataCacheFactory objects use it. However, connection pooling is specific to a cache client configuration on a single role instance. If you use multiple cache client configurations, each configuration maintains its own connection pool. If you scale out your role to run on multiple instances, each instance manages its own connection pool. These considerations are important for the Shared Caching connection quota.

Important
Connection pooling is not enabled by default when you programmatically configure a cache client. For more information, see Understanding and Managing Connections in Windows Azure.

Shared Caching

Caching (Preview) on Roles

<dataCacheClients>
  <tracing sinkType="DiagnosticSink" traceLevel="Error" />
  <dataCacheClient name="default" useLegacyProtocol="false" >
    <autoDiscover isEnabled="true" identifier="WebRole1" />
  </dataCacheClient>
</dataCacheClients>

Consider using compression for objects stored in the cache. Compression decreases the size of objects stored in the cache. The most obvious benefit is the lower memory requirements for storing the same number of items. However, there are also other potential benefits. Although compression requires additional client processor time, smaller objects are transported faster over the network, offsetting the compression costs. Smaller objects also reduce bandwidth, which is one of the Shared Caching quotas.

Before using compression, consider the type of object. For example, a .jpg or .mpg file is already compressed based on their object definition. Further compression requires processing time without significantly reducing the file size. Each scenario must be evaluated and potentially tested to understand the benefits of compression.

Compression can be enabled with the isCompressionEnabled attribute of the dataCacheClient section. The following example demonstrates how to enable compression in a configuration file.

<dataCacheClients>
  <dataCacheClient name="default" isCompressionEnabled="true">
    <!-- Other cache client configuration settings here -->
  </dataCacheClient>
</dataCacheClients>

Compression can also be enabled programmatically with the DataCacheFactory.IsCompressionEnabled property.

Tip
Even with compression techniques, it is recommended to cache small objects for optimal performance.

When using Shared Caching, use a cache that resides in the same Windows Azure datacenter as the cache clients. Cache requests across datacenters increases latency. In the same way, accessing a Windows Azure cache from on-premises code has unacceptable latency. This scenario is also not supported for production. If you require caching in an on-premises solution, consider Microsoft AppFabric 1.1 for Windows Server.

Role-based Caching automatically locates the cache on roles within a deployed cloud service. The cache in this architecture is even closer to the application than in the Shared Caching solution. Because of this improved locality, role-based Caching achieves higher performance overall than Shared Caching.

Like all network-based services, Windows Azure Caching has the potential to return transient errors. Therefore, it is important to include error handling and retry logic in your application.

Cache-specific errors are part of the DataCacheException class. The exception members, ErrorCode and SubStatus, determine the type of error received. These codes can be compared to the enumerations DataCacheErrorCode and DataCacheErrorSubStatus respectively.

The following table lists error codes and status codes that could require retries.

The previous errors are more common in retry scenarios, but they are not comprehensive. Use generic retry logic for Caching exceptions that encompass all possible errors. But you can vary the behavior based on the information here. For example, you can decide to skip retry attempts for the DataCacheErrorSubStatus.QuotaExceeded exception. Instead, you can perform some type of custom alert to an administrator while temporarily continuing without caching until the next quota hour.

With the Timeout error, it is also important to understand that the operation could have succeeded on the server. This situation can complicate the retry logic in some scenarios. For example, one overload of Put allows you to specify a version. If this operation succeeds but returns a Timeout error, simply retrying that call could fail due to the updated version on the server. In this example, you should get the object again with the new version before retrying the Put operation.

The actual code for retries can vary from a custom solution to a built-in framework. One example of a framework is the Microsoft Transient Fault Handling Application Block. The important thing is to develop a plan to handle, report, and recover from temporary failures. If there is a more permanent error condition, retries repeatedly fail and require additional troubleshooting.

There are several new methods and method overloads available for Caching hosted on Windows Azure roles. Most of these additions were made to the DataCache class. The following table lists new methods in the DataCache class.

There are also new overloads to the DataCache constructor that make it simpler to create a cache client. In the past, it was always necessary to create a DataCacheFactory object that returns the target cache. Now it is possible to create the cache with the DataCache constructor directly. The following example creates a client to the default cache from the default section of the configuration file.

DataCache cache = new DataCache();

The second overload specifies the named dataCacheClient configuration to use. The following example creates a client to the default cache from the dataCacheClient section of the configuration file named MyCacheClientSettings.

DataCache cache = new DataCache("MyCacheClientSettings");

The third overload specifies the named dataCacheClient configuration as well as the target cache name. The following example creates a client to the NamedCache1 cache from the default section of the configuration file.

DataCache cache = new DataCache("default", "NamedCache1");

In addition to these changes, there are also existing methods that previously applied only to Microsoft AppFabric 1.1 for Windows Server. Now these methods can be used with role-based Caching. These methods include any methods that handle regions, tagging, or notifications. For a complete discussion of the features now available in Caching, see Caching (Preview) on Roles.

Other Resources