Cloud Cache
Introducing the Windows Azure Caching Service
In a world where speed and scale are the key success metrics of any solution, they can’t be afterthoughts or retrofitted to an application architecture—speed and scale need to be made the core design principles when you’re still on the whiteboard architecting your application.
The Windows Azure Caching service provides the necessary building blocks to simplify these challenges without having to learn about deploying and managing another tier in your application architecture. In a nutshell, the Caching service is the elastic memory that your application needs for increasing its performance and throughput by offloading the pressure from the data tier and the distributed state so that your application is able to easily scale out the compute tier.
The Caching service was released as a Community Technology Preview (CTP) at the Microsoft Professional Developers Conference in 2010, and was refreshed in February 2011.
The Caching service is an important piece of the Windows Azure, and builds upon the Platform as a Service (PaaS) offerings that already make up the platform. The Caching service is based on the same code-base as Windows Server AppFabric Caching, and consequently it has a symmetric developer experience to the on-premises cache.
The Caching service offers developers the following capabilities:
- Pre-built ASP.NET providers for session state and page output caching, enabling acceleration of Web applications without having to modify application code
- Caches any managed object—no object size limits, no serialization costs for local caching
- Easily integrates into existing applications
- Consistent development model across both Windows Azure and Windows Server AppFabric
- Secured access and authorization provided by the Access Control service
While you can set up other caching technologies (such as memcached) on your own as instances in the cloud, you’d end up installing, configuring and managing the cache clusters and instances yourself. This defaults one of the main goals of the cloud, and PaaS in particular: to get away from managing these details. The Windows Server AppFabric Caching service removes this burden from you, while also accelerating the performance of ASP.NET Web applications running in Windows Azure with little or no code or configuration changes. We’ll show you how this is done throughout the remainder of the article.
Under the Hood
Now that you understand how caching can be a strategic design choice for your application, let’s dig a little deeper into what the Caching service really looks like. Let’s take an example of a typical shopping Web site, say an e-commerce site that sells Xbox and PC games, and use that to understand the various pieces of the Caching service.
At a high level, there are three key pieces to the puzzle:
- Cache client
- Caching service
- Connection between the client and the service
Cache client is the proxy that lives within your application—the shopping Web site in this case. It’s the piece of code that knows how to talk to the Caching service, in a language that both understand well. You need to include this client assembly in your application and deploy it with your Web application with the appropriate configuration to be able to discover and talk to the Caching service. (We’ll cover this process later in the article.)
Because there are some common application patterns, such as ASP.NET session state, where caching can be used out of the box, there are two ways to use the client.
For explicit programming against the cache APIs, include the cache client assembly in your application from the SDK and you can start making GET/PUT calls to store and retrieve data from the cache. This is a good way to store your games catalog or other reference data for your gaming Web site.
For higher-level scenarios that in turn use the cache, you need to include the ASP.NET session state provider for the Caching service and interact with the session state APIs instead of interacting with the caching APIs. The session state provider does the heavy lifting of calling the appropriate caching APIs to maintain the session state in the cache tier. This is a good way for you to store information like user preferences, shopping cart, game-browsing history and so on in the session state without writing a single line of cache code.
You can pretty much keep any object in the cache: text, data, blobs, CLR objects and so on. There’s no restriction on the size of the object, either. Hence, whether you’re storing explicit objects in cache or storing session state, the object size is not a consideration to choose whether you can use the Caching service in your application.
One thing to note about the Caching service is that it’s an explicit cache that you write to and have full control over. It’s not a transparent cache layer on top of your database or storage. This has the benefit of providing full control over what data gets stored and managed in the cache, but also means you have to program against the cache as a separate data store using the cache APIs.
This pattern is typically referred to as the cache-aside, where you first load data into the cache and then check if it exists there for retrieving and, only when it’s not available there, you explicitly read the data from the data tier. So, as a developer, you need to learn the cache programming model, the APIs, and common tips and tricks to make your usage of cache efficient.
One other thing to know about the cache client is the ability to cache a subset of the data that resides in the distributed cache servers, directly on the client—the Web server running the gaming Web site in our example. This feature is popularly referred to as the local cache, and it’s enabled with a simple configuration setting that allows you to specify the number of objects you wish to store and the timeout settings to invalidate the cache.
The second key part of the puzzle is the Caching service itself. Think of the Caching service as Microsoft running a large set of cache clusters for you, heavily optimized for performance, uptime, resiliency and scale out and just exposed as a simple network service with an endpoint for you to call. The Caching service is a highly available multitenant service with no management overhead for its users.
As a user, what you get is a secure Windows Communication Foundation (WCF) endpoint to talk to and the amount of usable memory you need for your application and APIs for the cache client to call in to store and retrieve data. The key here is usable memory. If you ask for 1GB cache memory, you get 1GB of usable memory to store your objects, unlike the amount of memory available on a Windows Azure instance you buy. The Caching service does the job of pooling in memory from the distributed cluster of machines it’s running and managing to provide the amount of usable memory you need. As a result, it also automatically provides the flexibility to scale up or down based on your cache needs with a simple change in the configuration. In that sense, think of the Caching service as a virtual pool of partitioned and shared memory that you can consume flexibly.
The other under-the-hood design point to understand is that the Caching service automatically partitions your cache so that you don’t lose data in the event of a machine going down, even if you haven’t bought the more expensive high availability (HA) option for your cache. (The HA option is not currently available in Windows Azure Caching, and only available on Windows Server AppFabric Caching today.) This partitioning scheme increases the performance and reduces the data loss probability automatically for you without ever having to learn about the back-end service.
The third and last part of the puzzle is the connection between your cache client and Caching service. The communication between the cache client and service is using WCF, and the WCF programming model is abstracted from the developer as the cache client translates GET/PUT calls to a WCF protocol that the service understands. The thing you need to know about this communication channel is that it’s secure, which is extremely critical, especially in the cloud world. Your cache is secured using an Access Control service token (which you get when you create the named cache). The Caching service uses this token to restrict access to your cached data by your client.
Architectural Guidance
We’ve spent a lot of time with customers who have varying degrees of application complexity . Usually, before we start whiteboarding the design choices and architecture, we’ll draw the simple diagram shown in Figure 1. For most cases this diagram captures the trade-offs between the three most basic elements involved in storing and manipulating data. (We’ve had some storage gurus argue with us that they can saturate network before they can max the disk throughput, hence we qualify the statement about “most” cases.)
.jpg "Memory, Network and Disk Usage in Data Manipulation Scenarios")
Figure 1 Memory, Network and Disk Usage in Data Manipulation Scenarios
The basic principle is that your memory on the local machine provides the fastest data access with lowest latency, but is limited to the amount of usable memory on the machine. As soon as you need more memory than your local machine can give you or you need to externalize the data from your compute tier (either for shared state or for more durability), the minimum price you pay is for the network hop. The slowest in this spectrum is storing data on a disk (solid-state drives help a little, but are still quite expensive).
Disk is the cheapest and largest store while memory is the most expensive and hence the most constrained in terms of capacity. The Caching service balances the various elements by providing the service as a network service to access a large chunk of distributed and shared memory across multiple compute tiers. At the same time, it provides an optimization with the local cache feature to enable a subset of the data to additionally reside on the local machine while removing the complexity of maintaining consistency between the local cache and the cache tier in the service.
With this background, let’s look at some top-level architectural considerations for using cache in your application.
What data should you put in the cache? The answer varies significantly with the overall design of your application. When we talk about data for caching scenarios, usually we break it into the data types and access patterns shown in Figure 2 (see msdn.microsoft.com/library/ee790832 for a deeper explanation of these data access patterns).
Figure 2 Data in Caching Scenarios
| Data Type | Access Pattern |
| Reference | Shared read |
| Activity | Exclusive write |
| Resource | Shared, concurrently read and written into, accessed by a large number of transactions |
Using this model to think about your data, you can plan for capacity and access patterns (for managing consistency, eviction, refreshes and so on) and restrict the data to the most frequently used or time-sensitive data used or generated by your application.
As an example, reference data should be readily partitioned into frequently accessed versus infrequently accessed to split between cache and storage. Resource data is a classic example where you want to fit as much as possible in cache to get the maximum scale and performance benefits. In addition to the cache tier, even the use of local cache on the client goes hand in hand with the data type. Reference data is a great candidate for keeping in the local cache or co-located with the client; while in most cases resource data can become quite chatty for the local cache due to frequent updates and hence best fits on the cache tier.
As the second-most-expensive-resource and usually the bottleneck for most inefficient implementations, network traffic patterns for accessing cache data is something to which you should pay particular attention. If you have a large number of small objects, and you don’t optimize for how frequently and how many objects you fetch, you can easily get your app to be network-bound. Using tags to fetch like data or using local cache to keep a large number of frequently accessed small objects is a great trade-off.
While the option to enable HA for a named cache is not yet available in the Caching service, it’s another factor to consider in your application design. Some developers and architects choose to use cache only as a transient cache. However, others have taken the leap of faith to move exclusively to storing a subset of their data (usually activity data) only in the cache by enabling the HA feature. HA does have a cost overhead, but it provides a design model that treats the cache as the only store of data, thereby eliminating the need to manage multiple data stores.
However, the cache is not a database! We cannot stress this point too strongly. The topic of HA usually makes it sound like the cache can replace your data tier. This is far from the truth—a SQL database is optimized for a different set of patterns than the cache tier is designed for. In most cases, both are needed and can be paired to provide the best performance and access patterns while keeping the costs low.
How about using the cache for data aggregation? This is a powerful yet usually overlooked scenario. In the cloud, apps often deal with data from various sources and the data not only needs to be aggregated but also normalized. The cache offers an efficient, high-performance alternative for storing and managing this aggregated data with high throughput normalization (in-memory as opposed to reading from and writing to disk), and the normalized data structure of the cache using key-value pairs is a great way to think about how to store and serve this aggregated data.
A common problem that application developers and architects have to deal with is the lack of guarantee that a client will always be routed to the same server that served the previous request. When these sessions can’t be sticky, you’ll need to decide what to store in session state and how to bounce requests between servers to work around the lack of sticky sessions. The cache offers a compelling alternative to storing any shared state across multiple compute nodes. (These nodes would be Web servers in this example, but the same issues apply to any shared compute tier scenario.) The shared state is consistently maintained automatically by the cache tier for access by all clients, and at the same time there’s no overhead or latency of having to write it to a disk (database or files).
The trade-off to consider here is that if you need ultra-low latency access to the shared state (session state in an online gaming Web site that tracks scores in real time, for example), then an external cache tier might not be your best option. For most other scenarios, using the cache tier is a quick yet powerful way to deal with this design pattern, which automatically eliminates the staleness from your data.
Be sure to spend enough time in capacity planning for your cache. Number of objects, size of each object, frequency of access of each object and pattern for accessing these objects are all critical in not only determining how much cache you need for your application, but also on which layers to optimize for (local cache, network, cache tier, using regions and tags, and so on). Remember that you can start with a simple configuration switch in your web.config file to start using caching without writing a single line of code, and you can spend years in designing a solution that uses caching in the most efficient way.
Setting Up Windows Azure Caching
To get started using the Windows Azure Caching service, head to portal.appfabriclabs.com. This is the CTP portal that you can use to get familiar with the Caching service. It doesn’t cost anything, but there are no service level agreements for the service.
On the portal, select the Cache option and then create a new cache by clicking New Namespace. The dialog box to configure the Cache service namespace is shown in Figure 3.
.jpg "Configuring a New Cache Service Namespace")
Figure 3 Configuring a New Cache Service Namespace
The only two options you need to specify are a unique service namespace and the cache size (of which you can choose between 128MB and 256MB in the CTP). When you choose OK the service will provision the cache for you in the background. This typically takes 10 to 15 seconds. When complete, you have a fully functional, distributed cache available to your applications.
Now that it’s created, you can take a look at the properties of your cache, shown in Figure 4. (Note that we’ve obfuscated some account-specific information here.)
.jpg "Cache Service Properties")
Figure 4 Cache Service Properties
You can see that we’ve created a cache using the namespace of CachingDemo. There are a few pieces of important information that you’ll want to grab, as we’ll use them later in our code: the Service URL and the Authentication Token. The Service URL is the TCP endpoint that your application will connect to when interacting with the Caching service. The Authentication Token is an encrypted token that you’ll pass along to Access Control to authenticate your service.
Caching in Your App
Now, before you start coding, download the Windows Azure SDK. You can find it at go.microsoft.com/fwlink/?LinkID=184288 or click the link on the portal.
Make sure you don’t have the Windows Server AppFabric Cache already installed on your machine. While the API is symmetric, the current assemblies are not compatible. The Windows Server AppFabric Cache registers its Caching assemblies in the Global Assembly Cache (GAC), so your application will load the wrong assemblies. This will be resolved by the time the service goes into production, but for now it causes a bit of friction.
To begin, let’s create a simple console application using C#. Once created, be sure to update the project so that it targets the full Microsoft .NET Framework instead of the Client Profile. You’ll also need to add the Caching assemblies, which can typically be found under C:\Program Files\Windows Azure SDK\V2.0\Assemblies\Cache. For now, add the following two assemblies:
- Microsoft.ApplicationServer.Caching.Client
- Microsoft.ApplicationServer.Caching.Core
One thing that changed between the October CTP and the February refresh is that you now must use the System.Security.SecureString for your authentication token. The purpose of SecureString is to keep you from putting your password or token into memory within your application, thereby keeping it more secure. However, to make this work in a simple console application, you’ll have to create the following method:
static private SecureString Secure(string token) {
SecureString secureString = new SecureString();
foreach (char c in token) {
secureString.AppendChar(c);
}
secureString.MakeReadOnly();
return secureString;
}
While this defeats the purpose of SecureString by forcing you to load the token into memory, it’s only used for this simple scenario.
Now, the next thing to do is write the code that will set us up to interact with the Caching service. The API uses a factory pattern, so we’ll have to define the factory, set some configuration settings and then load our default cache as shown in Figure 5.
Figure 5 Loading the Default Cache
private static DataCache configureDataCache(
SecureString authorizationToken, string serviceUrl) {
// Declare an array for the cache host
List<DataCacheServerEndpoint> server =
new List<DataCacheServerEndpoint>();
server.Add(new DataCacheServerEndpoint(serviceUrl, 22233));
// Set up the DataCacheFactory configuration
DataCacheFactoryConfiguration conf =
new DataCacheFactoryConfiguration();
conf.SecurityProperties =
new DataCacheSecurity(authorizationToken);
conf.Servers = server;
// Create the DataCacheFactory based on config settings
DataCacheFactory dataCacheFactory =
new DataCacheFactory(conf);
// Get the default cache client
DataCache dataCache = dataCacheFactory.GetDefaultCache();
// Return the default cache
return dataCache;
}
You can see that we’ve defined a new DataCacheServerEndpoint based on a service URL (that we’ll provide) that points to port 22233. We then create the DataCacheFactoryConfiguration, pass in our authentication token (which is a SecureString) and set it to the security properties—this will enable us to authenticate to the service. At this point it’s simply a matter of constructing the DataCacheFactory, getting the DataCache based on the default cache and returning the default cache.
While it’s not required to encapsulate this logic in its own method, it makes it much more convenient later on.
At this point, it’s pretty simple to pull this together in our application’s Main method (see Figure 6).
Figure 6 Console Application Main Method
static void Main(string[] args) {
// Hardcode your token and service url
SecureString authorizationToken =
createSecureString("YOURTOKEN");
string serviceUrl = "YOURCACHE.cache.appfabriclabs.com";
// Create and return the data cache
DataCache dataCache =
configureDataCache(authorizationToken, serviceUrl);
// Enter a value to store in the cache
Console.Write("Enter a value: ");
string value = Console.ReadLine();
// Put your value in the cache
dataCache.Put("key", value);
// Get your value out of the cache
string response = (string)dataCache.Get("key");
// Write the value
Console.WriteLine("Your value: " + response);
}
We provide the authentication token and service URL to our application, then pass them into the configureDataCache method, which sets the DataCache variable to the default cache. From here we can grab some input from the console, put it into the cache and then call Get on the key, which returns the value. This is a simple, but valid, test of the cache.
Including the tokens and service URL in the code is not ideal. Fortunately, the portal provides you with XML that you can insert within your app.config (or web.config) file, and the APIs will manage everything for you.
In the portal, select your cache, then click the View Client Configuration button. This opens a dialog that provides the configuration XML. Copy the XML snippet and paste it into your configuration file. The end result will look like Figure 7.
Figure 7 Client Configuration
<?xml version="1.0"?>
<configuration>
<configSections>
<section
name="dataCacheClient"
type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core"
allowLocation="true"
allowDefinition="Everywhere"/>
</configSections>
<dataCacheClient deployment="Simple">
<hosts>
<host
name="YOURCACHE.cache.appfabriclabs.com"
cachePort="22233" />
</hosts>
<securityProperties mode="Message">
<messageSecurity
authorizationInfo="YOURTOKEN">
</messageSecurity>
</securityProperties>
</dataCacheClient>
</configuration>
Now we can heavily refactor our code, get rid of the createSecureString and configureDataCache methods, and be left with this:
static void Main(string[] args) {
DataCacheFactory dataCacheFactory =
new DataCacheFactory();
DataCache dataCache = dataCacheFactory.GetDefaultCache();
Console.Write("Enter a value: ");
string value = Console.ReadLine();
dataCache.Put("key", value);
string response = (string)dataCache.Get("key");
Console.WriteLine("Your value: " + response);
}
You can see that all we have to do is create a new instance of the DataCacheFactory, and all the configuration settings in the
app.config file are read in by default.
As you’ve seen, you can use the APIs directly or manage DataCacheFactory in your configuration. While we’ve only performed PUT and GET operations on simple data, we could easily store data retrieved from SQL Azure, Windows Azure or another provider of data. For a more complex look at using the cache for reference data stored in SQL Azure, refer to the Caching Service Hands-On Lab in the Windows Azure Training Course (msdn.microsoft.com/gg457894).
Storing Session Data
Next, let’s take a look at how to use the Caching service to store the session data for our ASP.NET Web application. This is a powerful technique, as it allows us to separate the session state from the in-process memory of each of our Web clients, thus making it easy to scale our applications beyond one instance in Windows Azure.
This is important for servicesthat don’t support sticky sessions, such as Windows Azure. There’s no way to guarantee that a user will hit the same instance with each additional request—in fact, the Windows Azure load balancer explicitly uses a round-robin approach to load balancing, so it’s likely that your user will hit a new instance. By using the Caching service for session state, it doesn’t matter which instance your user hits because all the instances are backed by the same session state provider.
To begin, create a new Windows Azure Project and add an ASP.NET Web Role. In the Web Role, add all of the assemblies provided by the Windows Azure SDK, including:
- Microsoft.ApplicationService.Caching.Client
- Microsoft.ApplicationService.Caching.Core
- Microsoft.Web.DistributedCache
- Microsoft.WindowsFabric.Common
- Microsoft.WindowsFabric.Data.Common
Next, update your web.config file so that the very first things to follow the <configuration> element are the <configSections> and <dataCacheClient> elements (you’ll receive errors if they aren’t the first elements).
Now, the key to using the Caching service for session state is the Microsoft.Web.DistributedCache assembly. It contains the custom session state provider that uses the Caching service. Return to the LABS portal where you grabbed the XML for your configuration files, and find the <sessionState> element—you can place this directly in the <system.web> element of your web.config file and it will immediately tell your application to start leveraging the Caching service for session state:
<system.web>
<sessionState mode="Custom"
customProvider="AppFabricCacheSessionStoreProvider">
<providers>
<add name="AppFabricCacheSessionStoreProvider"
type="Microsoft.Web.DistributedCache.DistributedCacheSessionStateStoreProvider, Microsoft.Web.DistributedCache"
cacheName="default"
useBlobMode="false" />
</providers>
</sessionState>
...
</system.web>
To validate that this is working, open the Global.asax.cs file and add the following code to the Session_Start method:
void Session_Start(object sender, EventArgs e) {
int i = 0;
while (i < 10) {
Session.Add(Guid.NewGuid().ToString(), DateTime.Now.ToString());
i++;
}
}
This will add 10 random items into your session context. Next, open up your Default.aspx.cs page and update the Page_Load method:
protected void Page_Load(object sender, EventArgs e) {
foreach (var key in Session.Contents) {
Response.Write("key: " + key + ", guid: " +
Session[key.ToString()].ToString() + "<br/>");
}
}
This will write out all the values that were added into the session context. Finally, open the ServiceConfiguration.cscfg file and increase the instance count from 1 to 2:
Now, when you hit F5, you’ll get two instances of your application running in the Compute Emulator. Notice that no matter how many times you refresh the page, you’ll always have the same 10 values in your session state—this is because it’s a shared session, and session start only runs once. Conversely, if you don’t use the Caching service as your session state provides, but opt to keep the default in-process choice, you’ll have different values on each of the instances.
What’s Next?
Windows Azure Caching service is slated to go into production as a commercial service in the first half of 2011. In the first commercial release, some of the features that are available in Windows Server AppFabric will not yet be available. Some of these exclusions are purposeful, as they may not apply in the cloud world. However, features like notifications are relevant in Windows Azure as well and are critical in completing the local-cache scenario, and hence are part of the short-term roadmap for Microsoft. Similarly, the option of turning on HA for a given named cache as a premium capability is also high on the priority list.
The growing popularity of Windows Server AppFabric Caching has resulted in a number of new feature requests that open up the applicability of caching to an even broader set of scenarios. Some of the capabilities that are being discussed include the ability to perform rich queries on the cache and enabling an easier way to retrieve bulk data from a named cache.
In addition, the success of the Caching session state provider scenarios with ASP.NET has resulted in requests for the ability to associate write-behind and read-through queries with the cache so that the cache can become the primary way to manipulate data, while letting the associated queries update the data tier in the back end.
We’ll be evaluating these and other features for possible inclusion in future releases of Windows Azure Caching. In the meantime, we encourage you to experiment with the current Caching service implementation and let us know how it works for you.
Karandeep Anand is a principal group program Manager with the Windows Azure product group at Microsoft. His team is responsible for building the next generation application platform and services for the Windows Azure and Windows Server. You can reach him at Karandeep.Anand@microsoft.com.
Wade Wegner is a technical evangelist at Microsoft, responsible for influencing and driving the Microsoft technical strategy for the Windows Azure. You can reach him through his blog at wadewegner.com or on Twitter at twitter.com/WadeWegner.
Comments (6)
