Using a controller to manage static resources
Share
If you’re like me, you’re more or less addicted to YSlow and getting the best performance for your web applications whenever you can. One challenge is to address multiple YSlow recommendations at the same time, particularly when it asks you to compress the content sent to clients, and in the same breath asks that ETags are used to avoid resending cached content for particular resources. In this excellent overview of some of the pitfalls of using ETags, you’ll learn that if you try to base your ETag value on compressed content, the timestamp-based format of that content will cause a new ETag to generate for every resource request, and the end result is a constant fetch of the resource, breaking your caching scheme.
For this challenge I decided to use a Controller, and mapped its route so that a request for the Site.css file in my “/Content/” folder, where the default project template stores CSS files (and where I store all static content), is declared in markup as “/static/Site.css”. Not a big difference for me, but I can customize the controller’s behavior later to search in multiple places, hiding the details behind the static URL path.
In ASP.NET MVC you have a few options for getting between the pipeline to fine tune your requests and responses. In this post I want to outline a static resource controller I’m using to ensure that my static content is cached on both the client and the server, is sent compressed whenever possible, and makes appropriate use of ETags. Your static resource controller should do the following things for you:
- It should cache content on the client, and know when to send a 304 – Not Modified whenever the request is asking for an ETag that maps to the file it’s looking for, or if the last modified date of the resource file hasn’t changed since the last request. I prefer to evaluate the ETag first, just in case the file’s last modified date was changed, but the content of the file itself is identical, since that would result in an unnecessary cache invalidation.
- It should cache content on the server, ensuring multiple requests from different browsers are handled efficiently (in this case from memory rather than from disk), and know when to invalidate the cached copy as soon as the underlying resource changes. This will help when you are lucky enough to have a traffic problem.
- It should compress the content returned to the client based on what the client is able is to accept.
- It should work with GZIP and DEFLATE, and the ETags should take these into account so that the controller works even when compression is turned off.
Here’s the skeleton version of the controller, with a few missing methods, but enough to read through to see how an HTTP GET call using the controller handles static resources.
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Web;
using System.Web.Caching;
using System.Web.Mvc;
using Dimebrain.Commons.Mvc.Extensions;
using Dimebrain.Commons.Mvc.Filters;
namespace Dimebrain.Commons.Mvc.Controllers
{
[CompressFilter]
public class StaticResourceController : Controller
{
[AcceptVerbs(HttpVerbs.Get)]
public void Get(string file)
{
var relativePath = string.Format("~/Content/{0}", file);
var absolutePath = Server.MapPath(relativePath);
// 304 (If-None-Match), a better test of uniqueness than modified date
var etag = GenerateETag(absolutePath);
if (BrowserIsRequestingFileIdentifiedBy(etag))
{
Response.StatusCode = (int)HttpStatusCode.NotModified;
Response.SuppressContent = true;
return;
}
// 404 (NotFound), if ETag differs, I/O access is inevitable
if (!System.IO.File.Exists(absolutePath))
{
Response.StatusCode = (int)HttpStatusCode.NotFound;
Response.SuppressContent = true;
return;
}
// 304 (If-Last-Modified)
var lastModified = new FileInfo(absolutePath).LastWriteTime;
if(BrowserIsRequestingFileUnmodifiedSince(lastModified))
{
Response.StatusCode = (int) HttpStatusCode.NotModified;
Response.SuppressContent = true;
return;
}
// 200 - OK
CacheOnClient(etag, lastModified);
var content = CacheOrFetchFromServer(relativePath, absolutePath);
using (var sw = new StreamWriter(Response.OutputStream))
{
sw.Write(content);
}
var contentType = absolutePath.MimeType();
Response.ContentType = contentType;
}
}
}
That’s the controller in a nutshell. It compresses the stream serving the content, and it gives the browser multiple opportunities to use its cached copy, and it also caches the content on the server. Looking closer at the supporting methods of the controller, you’ll find that both the ETag and the server cache dependency on based on the file itself; when it changes, a new value is generated for the ETag, and the server cache is invalidated. This is a great way to set this controller up and forget about it, because whenever you make a change to the underlying resource, all bets are off and the browser is guaranteed to get the latest version on the next request.
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Web;
using System.Web.Caching;
using System.Web.Mvc;
using Dimebrain.Commons.Mvc.Extensions;
using Dimebrain.Commons.Mvc.Filters;
namespace Dimebrain.Commons.Mvc.Controllers
{
[CompressFilter]
public class StaticResourceController : Controller
{
private const string DeflateTag = "-deflate";
private const string GzipTag = "-gzip";
private const string LastModifiedSinceHeader = "If-Modified-Since";
private const string IfNoneMatchHeader = "If-None-Match";
private const string DefaultEncodingCodePage = "iso-8859-1";
[AcceptVerbs(HttpVerbs.Get)]
public void Get(string file)
{
var relativePath = string.Format("~/Content/{0}", file);
var absolutePath = Server.MapPath(relativePath);
// 304 (If-None-Match), a better test of uniqueness than modified date
var etag = GenerateETag(absolutePath);
if (BrowserIsRequestingFileIdentifiedBy(etag))
{
Response.StatusCode = (int)HttpStatusCode.NotModified;
Response.SuppressContent = true;
return;
}
// 404 (NotFound)
if (!System.IO.File.Exists(absolutePath))
{
Response.StatusCode = (int)HttpStatusCode.NotFound;
Response.SuppressContent = true;
return;
}
// 304 (If-Last-Modified)
var lastModified = new FileInfo(absolutePath).LastWriteTime;
if(BrowserIsRequestingFileUnmodifiedSince(lastModified))
{
Response.StatusCode = (int) HttpStatusCode.NotModified;
Response.SuppressContent = true;
return;
}
// 200 - OK
CacheOnClient(etag, lastModified);
var content = CacheOrFetchFromServer(relativePath, absolutePath);
using (var sw = new StreamWriter(Response.OutputStream))
{
sw.Write(content);
}
var contentType = absolutePath.MimeType();
Response.ContentType = contentType;
}
private bool BrowserIsRequestingFileUnmodifiedSince(DateTime lastModified)
{
if (Request.Headers[LastModifiedSinceHeader] == null)
{
return false;
}
// Header values may have additional attributes separated by semi-colons
var ifModifiedSince = Request.Headers[LastModifiedSinceHeader];
if (ifModifiedSince.IndexOf(";") > -1)
{
ifModifiedSince = ifModifiedSince.Split(';').First();
}
// Get the dates for comparison; truncate milliseconds in date if needed
var sinceDate = Convert.ToDateTime(ifModifiedSince).ToUniversalTime();
var fileDate = lastModified.ToUniversalTime();
if (sinceDate.Millisecond.Equals(0))
{
fileDate = new DateTime(fileDate.Year,
fileDate.Month,
fileDate.Day,
fileDate.Hour,
fileDate.Minute,
fileDate.Second,
0);
}
return fileDate.BeforeOrEqual(sinceDate);
}
private bool BrowserIsRequestingFileIdentifiedBy(string etag)
{
if (Request.Headers[IfNoneMatchHeader] == null)
{
return false;
}
var ifNoneMatch = Request.Headers[IfNoneMatchHeader];
return ifNoneMatch.EqualsIgnoreCase(etag);
}
private static string CacheOrFetchFromServer(string relativePath, string absolutePath)
{
var cache = HttpRuntime.Cache;
string content;
if (cache[relativePath] == null)
{
var encoding = Encoding.GetEncoding(DefaultEncodingCodePage);
var dependency = new CacheDependency(absolutePath);
content = System.IO.File.ReadAllText(absolutePath, encoding);
cache.Insert(relativePath, content, dependency);
}
else
{
content = cache[relativePath].ToString();
}
return content;
}
private bool HasGzipFilter
{
// You can't inspect the response headers directly in IIS6 / Cassini
get { return Convert.ToBoolean(HttpContext.Items["gzipped"] ?? "false"); }
}
private bool HasDeflateFilter
{
// You can't inspect the response headers directly in IIS6 / Cassini
get { return Convert.ToBoolean(HttpContext.Items["deflated"] ?? "false"); }
}
private void CacheOnClient(string etag, DateTime lastModified)
{
var futureDate = 365.Days();
var expires = futureDate.FromNow();
var cache = Response.Cache;
// Cacheability must be set to public for SetETag to work; you could also
// add the ETag header yourself with AppendHeader or AddHeader methods
cache.SetCacheability(HttpCacheability.Public);
cache.AppendCacheExtension("must-revalidate, proxy-revalidate");
cache.SetExpires(expires);
cache.SetMaxAge(futureDate);
cache.SetLastModified(lastModified);
cache.SetETag(etag);
}
private string GenerateETag(string absolutePath)
{
var sb = new StringBuilder();
sb.Append(absolutePath.MD5());
if(HasGzipFilter)
{
sb.Append(GzipTag);
}
else if(HasDeflateFilter)
{
sb.Append(DeflateTag);
}
return sb.ToString();
}
}
}
You can see how the Cassini workaround is used in the code above, to check if either GZIP or DEFLATE was used based on the results of compression. If it was, the ETag is appended with the appropriate value, so that a browser that doesn’t support GZIP (and will rename nameless to protect the guilty) can still participate in ETags but not accidentally receive a compressed version, which would happen if only the MD5 was used to identify the file. You’re also throwing everything but the kitchen sink at the client cache, giving it more than enough information to work with caching. Keep in mind that it’s still up to your code to handle telling the browser that there’s nothing to see here, and that it’s time to move along, by comparing “If-Modified-Since” against the resource’s last modified date, and “If-None-Match” against the current ETag generated for the resource, to see if there are any discrepancies. There’s a few gotchas in this process, such as ensuring you truncate milliseconds in your file’s last modified timestamp, if the browser does the same thing, otherwise you’ll continue to send the same cached file every time.
To set up compression, you can use an ActionFilterAttribute, overriding the OnActionExecuting event to spin up a compression filter and add it to the output stream. The reason you would run compression prior to the action completing in this case is because I personally prefer to use the Visual Studio Development Server, aka Cassini, when debugging my applications. Unfortunately, Cassini emulates II6 which does not support integrated pipeline mode, and will cause exceptions when you try to inspect response headers directly when asking questions like “Is this response compressed?”, so, I apply compression before my controller action executes, and toss the results of the compression filter into HttpContext.Items so it’s available for the action down the line. You don’t have to do this if you’re debugging on a local IIS7 instance, or if you’re comfortable doing it live. Here’s the code to add compression to the controller through a filter.
using System.IO.Compression;
using System.Web.Mvc;
namespace Dimebrain.Commons.Mvc.Filters
{
public class CompressFilterAttribute : ActionFilterAttribute
{
private const string ContentEncodingHeader = "Content-Encoding";
private const string AcceptEncodingHeader = "Accept-Encoding";
private const string GZipValue = "gzip";
private const string DeflateValue = "deflate";
public override void OnActionExecuting(ActionExecutingContext filterContext)
{
var request = filterContext.HttpContext.Request;
var acceptEncoding = request.Headers[AcceptEncodingHeader];
if (string.IsNullOrEmpty(acceptEncoding)) return;
acceptEncoding = acceptEncoding.ToLowerInvariant();
var response = filterContext.HttpContext.Response;
if (acceptEncoding.Contains(GZipValue))
{
response.AppendHeader(ContentEncodingHeader, GZipValue);
response.Filter = new GZipStream(response.Filter, CompressionMode.Compress);
// Hack so we can continue to use Cassini
filterContext.HttpContext.Items["gzipped"] = "true";
}
else if (acceptEncoding.Contains(DeflateValue))
{
response.AppendHeader(ContentEncodingHeader, DeflateValue);
response.Filter = new DeflateStream(response.Filter, CompressionMode.Compress);
// Hack so we can continue to use Cassini
filterContext.HttpContext.Items["deflated"] = "true";
}
}
}
}
The last thing you need to do to get this working is declare the controller route wherever you’re doing such things, which by default is in your Global.asax file’s Application_Start event handler (via the RegisterRoutes method).
// Static Resources
routes.Add(new Route
("static/{file}", new MvcRouteHandler())
{
Defaults = new RouteValueDictionary(
new { controller = "StaticResource",
action = "Get",
file = ""
})
});
Now you can declare your static resources that live in the Content folder like this, and get all the benefits of the controller.
That’s the code. The last snippet shows you all of the extension methods I used to make the example more readable. Hopefully this will feed your YSlow addiction just a little bit more, but more importantly, work effectively with cached static content in your MVC projects.
public static string MD5(this string filePath)
{
var sb = new StringBuilder();
using (var fs = new FileStream(filePath, FileMode.Open))
{
using (var br = new BinaryReader(fs))
{
var md5 = new MD5CryptoServiceProvider();
var hash = md5.ComputeHash(br.BaseStream);
foreach (var hex in hash)
{
sb.Append(hex.ToString("x2"));
}
return sb.ToString();
}
}
}
public static string MimeType(this string filePath)
{
const string key = "Content Type";
var extension = Path.GetExtension(filePath).ToLowerInvariant();
var registryKey = Registry.ClassesRoot.OpenSubKey(extension);
return registryKey != null && registryKey.GetValue(key) != null
? registryKey.GetValue(key).ToString()
: "application/octet-stream";
}
public static bool EqualsIgnoreCase(this string left, string right)
{
return String.Compare(left, right, true) == 0;
}
public static TimeSpan Days(this int value)
{
return TimeSpan.FromDays(value);
}
public static DateTime FromNow(this TimeSpan value)
{
return DateTime.UtcNow.Add(value);
}
public static bool BeforeOrEqual(this DateTime left, DateTime right)
{
return left.CompareTo(right) <= 0;
}
Socialized