Fiddler PowerToy - Part 2: HTTP Performance

  • Article

 

Eric Lawrence
Microsoft Corporation

June 2005

Applies to:
   Microsoft Internet Explorer
   Microsoft .NET Framework
   Microsoft Fiddler PowerToy

Summary: Learn how to build a faster Web site using the Microsoft Fiddler HTTP Debugger. In this article, we'll use Fiddler to explore HTTP performance, caching, and compression. (12 printed pages)

Prerequisite: If you have not yet installed and configured Microsoft Fiddler, please begin with the first installment in this series: Part 1: HTTP Debugging.

Contents

HTTP Performance: An Overview
Tweaking "First Visit" Performance
Introduction to HTTP Caching
Conditional Requests and the WinInet Cache
Introduction to HTTP Compression
Winning the Performance Battle
To Be Continued...

HTTP Performance: An Overview

It's no secret—users love fast Web sites. Users are notoriously impatient, and unless your Web site has no competitive substitute, users are unlikely to stick around if your site's performance doesn't measure up. If your site has visitors from around the world, ensuring your site operates efficiently is even more critical, as international network connections generally suffer from the twin banes of snappy sites: high latency and low bandwidth.

There are many options for improving your site's performance: compression, caching, geographic load balancing, adding hardware, and so forth. Optimizing the use of compression and caching is often the best place to start, as configuration changes are generally free and can return dramatic benefits.

In this article, we'll use the Microsoft Fiddler HTTP Debugger to explore HTTP performance, caching, and compression.

Tweaking "First Visit" Performance

On their crucial first visit to your site, visitors must download every piece of content used to generate the page, including JScript, CSS, images, and HTML. If your page is too slow to load, visitors may leave your page before it's even done downloading!

By exposing all HTTP traffic, Fiddler readily shows which files are used to generate a given page. Shift+click multiple entries in the HTTP Sessions list to calculate the "total page weight"—the number of requests and the bytes transferred.

Figure 1. Fiddler's Performance Statistics View

The best way to ensure a "Wow, this is fast" first impression is to deliver fewer and smaller files.

Tips for fast first-visits:

  • Use fewer graphics.
  • Extract styles into a single CSS file.
  • Extract script blocks into a single JS file.
  • Simplify your page layout.
  • Use HTTP Compression.

Once you've tuned your site for a fast first visit, you can make it even faster for return visitors by taking advantage of HTTP caching.

Introduction to HTTP Caching

Two key factors in improving the speed of your Web applications are:

  • Reducing the number of request/response roundtrips.
  • Reducing the number of bytes transferred between the server and the client.

HTTP caching is of the best ways to reduce roundtrips and bytes transferred. Caching provides a mechanism for a client or proxy to store HTTP responses for later use, so that requests need not cross the network.

Other than performance, another benefit of maximizing use of HTTP caching comes from the fact that bandwidth isn't free. By tuning caching for a major Microsoft site, we were able to reduce our outbound bandwidth costs by over $10,000 per month.

To enhance performance, Microsoft Internet Explorer and other Web clients maintain a local cache of resources downloaded from remote Web servers.

When a resource is needed by the client, there are three possible actions:

  • Send a plain HTTP request to the remote Web server asking for a resource
  • Send a conditional HTTP request to the origin server asking for the resource only if it differs from the locally cached version
  • Use a locally cached version of the resource, if a cached copy is available

When sending a request, the client may use one of the following headers:

Table 1. Client Cache Headers

Pragma: no-cache The client is unwilling to accept any cached responses from caches along the route and the origin server must be contacted for a fresh copy of the resource.
If-Modified-Since: datetime The server should return the requested resource only if the resource has been modified since the date-time provided by the client.
If-None-Match: etagvalue The server should return the requested resource if the ETAG of the resource is different than the value provided by the client. An ETAG is a unique identifier representing a particular version of a file.

A client indicates that it has a cached response available for use by sending a "Conditional request" containing the headers If-Modified-Since or If-None-Match. If the server replies to a conditional request with HTTP/304 Not Modified, the client is directed to reuse its cached response. Otherwise, the server should return a new response and the client should discard its outdated cache item.

Observe two consecutive requests for an image file in the following code sessions. In the first session, no locally cached version of the file is present, so the server returns the file along with an ETAG value and the date-time of the last modification of the file. In the subsequent session, a locally cached version of the file is now available, so a conditional request is made, passing up the ETAG of the cached response as well as the Last-Modified time of the original request. Since the image has not changed since the cached version (either because the ETAG matches or the If-Modified-Since value matches the Last-Modified value) the server returns a 304 to the client to direct it to use the cached response.

Session #1

GET /images/banner.jpg HTTP/1.1
Host: www.bayden.com

HTTP/1.1 200 OK
Date: Tue, 08 Mar 2005 00:32:46 GMT
Content-Length: 6171
Content-Type: image/jpeg
ETag: "40c7f76e8d30c31:2fe20"
Last-Modified: Thu, 12 Jun 2003 02:50:50 GMT

Session #2

GET /images/banner.jpg HTTP/1.1
If-Modified-Since: Thu, 12 Jun 2003 02:50:50 GMT
If-None-Match: "40c7f76e8d30c31:2fe20"
Host: www.bayden.com

HTTP/1.1 304 Not Modified

Because an HTTP/304 response contains only headers and no body, it crosses the network much more quickly than if the full resource had been re-downloaded. However, even an HTTP/304 requires a full roundtrip to the remote Web server; by carefully setting response headers, a Web application developer can eliminate the need to issue even conditional requests.

Generally, the cacheability of an HTTP response is controlled by headers sent in the response. The HTTP specification describes the headers that control caching. The optional Cache-Control and Expires headers are the primary mechanisms for a Web server to indicate to a proxy or a client how content may be cached.

The Expires header contains an absolute date-time after which a cached copy of a response should no longer be considered fresh. If the Expires header contains something other than a date (0 or -1 are common values), the response should immediately be treated as stale. A fresh cache entry may be reused without contacting the server again; a stale cache entry should not be reused without first contacting the Web server to ensure that it is still up-to-date.

For example, let's look at the previous example, except we'll add an Expires header to the first response:

Session #1

GET /images/banner.jpg HTTP/1.1
Host: www.bayden.com

HTTP/1.1 200 OK
Date: Tue, 08 Mar 2005 00:32:46 GMT
Content-Length: 6171
Content-Type: image/jpeg
Expires: Tue, 12 Jun 2007 02:50:50 GMT
Last-Modified: Thu, 12 Jun 2003 02:50:50 GMT

Session #2

<no HTTP request is made; cached version is used automatically>

As you can see, we've improved performance by adding an Expires header, since no conditional HTTP request is made during Session #2.

Similarly, the Cache-Control header contains a list of tokens that control caching. Any Cache-Control directives supersede the Expires header.

Commonly used Cache-Control tokens include those found in table 2.

Table 2. Common Cache-Control Headers

Value Meaning
public The response may be stored in any cache, including caches shared among many users.
private The response may only be stored in a private cache used by a single user.
no-cache The response should not be reused to satisfy future requests.
no-store The response should not be reused to satisfy future requests, and should not be written to disk. This is primarily used as a security measure for sensitive responses.
max-age=#seconds The response may be reused to satisfy future requests within a certain number of seconds.
must-revalidate The response may be reused to satisfy future requests, but the origin server should first be contacted to verify that the response is still fresh.

Notes

  • You can learn more about how to configure caching for IIS from the Microsoft Knowledge Base article How to Modify the Cache-Control HTTP Header When You Use IIS.
  • You can learn more about how to configure caching for ASP.NET pages the Microsoft Knowledge Base article How To Cache in ASP.NET by Using Visual C# .NET.
  • If you find that you often update files on your Web site without changing the filenames, take care to set appropriate cache lifetimes. For instance, if you want thisyear.gif to show the current year on your site, you need to make sure the caching directives don't call for an expiration longer than 1 day, or else a user visiting on December 31st won't see the right image when they go back on January 1st!
  • For legacy reasons, servers may send a Pragma: no-cache header. This is treated as Cache-Control: no-cache.
  • The Vary header signals to a cache that a response is valid for reuse only under certain circumstances. For example, use Vary: User-Agent to direct that the current response may only be reused for future requests sending the same User-Agent header. The directive Vary: * is equivalent to Cache-Control: no-cache.

Using the HTTP Sessions list, Fiddler users can see whether pages contain HTTP Caching headers.

Figure 2. Fiddler Sessions List

If a response does not contain Expires or Cache-Control headers, the client may be forced to issue a conditional request to ensure that the resource is still fresh.

Conditional Requests and the WinInet Cache

Internet Explorer takes advantage of the caching services provided by Microsoft Windows Internet Services (WinInet).

WinInet allows the user to configure the size and behavior of the cache. To access the cache settings:

  1. Open Internet Explorer.
  2. On the Tools menu, choose Internet Options.
  3. On the General tab, in the Temporary Internet Files box, click Settings.

At the top of the Settings dialog box, there are four choices.

Figure 3. Internet Explorer Cache Options

The vast majority of users leave the setting at the default of Automatically.

The most important fact to keep in mind is that these four options mostly impact the behavior when there are no caching headers on the HTTP responses; when caching headers are present, Internet Explorer will always respect them. The following table describes the impact of these settings on request behavior.

Table 3. Cache behaviors

Setting Cache copy is fresh Cache stale No cache-directives were present
Every visit to the page No request Conditional request Conditional request
Every time you start Internet Explorer No request Conditional request Conditional request
Automatically No request Conditional request Heuristic (see below)
Never No request Conditional request No request

Cached content is considered fresh if the request is made during the freshness lifetime specified by the Cache-Control or Expires headers on the original response. Cached content is considered stale if the request is made after the end of the freshness lifetime specified.

The Automatically setting bears some explanation—how can WinInet know if the cached resource is fresh when no caching directives were provided on the server's HTTP response?

The answer is that WinInet can't know for sure and a Heuristic process is followed to make a "best guess" effort. In the Automatically state, the Heuristic will issue a conditional request unless all of the following criteria are met:

  • The cached resource bears a Content-Type that begins with image/.
  • The cached resource has a Last-Modified time.
  • The URL to the cached resource does not contain a question mark (hinting that it's a CGI request).
  • The cached resource has been conditionally requested at least once within the most recent 25 percent of its overall age in the cache.

If all of the criteria above are met, no request is made.

As a Web developer, you should always ensure that you send appropriate caching headers to guarantee you get optimum cache behaviors.

Flagging Performance Problems

You can use Fiddler's Custom Rules to draw attention to potential performance problems. For instance, you can flag any response larger than 25KB.

To add this rule, click Rules and then Custom Rules, and add the following code inside the OnBeforeResponse event handler:

// Flag files over 25KB
   if (oSession.responseBodyBytes.length > 25000){
      oSession["ui-color"] = "red";
      oSession["ui-bold"] = "true";
      oSession["ui-customcolumn"] = "Large file";
   }

Similarly, you can flag responses that do not specify caching information:

   // Mark files which do not have caching information
if (!oSession.oResponse.headers.Exists("Expires") &&
!oSession.oResponse.headers.Exists("Cache-Control")){
      
oSession["ui-color"] = "purple";
      oSession["ui-bold"] = "true";
   }

Introduction to HTTP Compression

All popular Web servers and browsers offer support for HTTP Compression. HTTP Compression can dramatically decrease the number of bytes that are transmitted between the server and the client; savings of over 50 percent for HTML, XML, CSS, and JS are common.

A Web browser signals to the server that it is willing to accept HTTP compressed content by listing the supported compression types in the request headers. For instance, consider the following request to the new MSN Search homepage:

GET / HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
Host: search.msn.com

The Accept-Encoding header indicates Internet Explorer is willing to accept responses that have been compressed using either the GZIP or DEFLATE formats.

The MSN Search server obligingly returns the compressed contents; the Content-Encoding response header indicates the GZIP format was used:

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/6.0 —Microsoft-HTTPAPI/1.0
X-Powered-By: ASP.NET
Vary: Accept-Encoding
Content-Encoding: gzip
Date: Tue, 15 Feb 2005 09:14:36 GMT
Content-Length: 1277
Connection: close
Cache-Control: private, max-age=3600

Using Fiddler, you can decompress the response using the Transformer tool on the Session Inspector tab.

Figure 4. Transformer Inspector before decompressing GZIP'd response

Click the No Compression radio button to decompress the response inside Fiddler. Compression reduced the number of bytes transferred by over 57 percent.

Figure 5. Transformer Inspector after decompressing response

The savings are more dramatic for the common.css file used by the MSN Search homepage; the CSS file was compressed 81 percent (from 25,288 bytes to 4,648 bytes). Note that image files like GIFs, JPEGs, and PNGs generally are already compressed and thus usually are not delivered with HTTP compression.

You can use Fiddler to simulate HTTP compression by checking "Simulate GZIP Compression" on the Fiddler Rules menu.

Enabling compression for static files in IIS has a minimal CPU impact on IIS Web servers, because the files are compressed only the first time and then cached on the server. Enabling compression for dynamic files like ASP.NET pages may impact your servers' CPU performance; you'll want to evaluate this performance impact before enabling dynamic compression on production Web servers.

Winning the Performance Battle

Improving HTTP efficiency is only half of the challenge; if your Web application itself is slow, it won't matter how efficient your HTTP traffic is. Read Ten Tips for Writing High-Performance Web Applications to learn more about performance optimization.

To Be Continued...

In this installment, we've covered HTTP performance and how to flag HTTP performance issues using Fiddler.

In future installments, we'll continue to explore how Microsoft Fiddler can help you build better Web applications. Please feel free to suggest new topics for this series using the Help | Send Feedback option inside Fiddler.

Note: Fiddler version 0.9.9 is now available. Also available is an early beta version of the Fiddler Script Editor, a syntax-aware script-editing environment.

 

About the author

Eric Lawrence is a program manager on the Internet Explorer team. Before joining the Internet Explorer team, Eric worked on Microsoft Office Online and as an intern Program Manager on Microsoft SharePoint Products and Technologies.