mark nottingham

Fascinating! Are these RFC drafts presented to the IETF HTTP mailing list?

or will it be put into the new rewritten HTTP spec ? you said the rewriting involves writing down all the known stuff/accepted practices …

re: configuration vs. header – I’ve been thinking that more HTTP libraries should emphasize the request object, and use the headers as a kind of API. So, setting stale-while-revalidate is a reasonable option for the HTTP library, but also fits into the HTTP message itself. The same applies to stale-if-error. And I think, though I haven’t thought deeply about it, sending that cache control to the library and then possibly over the wire to whatever other intermediate cache’s exist, seems ok and maybe good.

Sweet….. this is exactly what I need for Tailrank.

Right now we prefetch but as you noted it’s prone to problems

The one you DIDN’T mention is that if you have LOTs of URLs you might not KNOW what you need to revalidate.

…. so prefetch doesn’t work here.

The Cache-Control header changeset is now in Squid-2, and all of this will be in the upcoming 2.7 release.

One interesting thing that occurs to me is that if you don’t prefetch at all for user-facing HTML pages, you could serve up a fake “loading / progress bar” page as the blank-slate result for the first unfortunate users to make the request.

Kick off the async refresh, and leave a sanely timed meta-refresh on the blank-slate page, and hopefully they’re happy enough not to complain.

IE has implemented a couple of relevant cache-control extensions since IE5, not sure how widely they are used:

http://msdn2.microsoft.com/en-us/library/ms533020.aspx#Use_Cache-Control_Extensions

This is good stuff. My colleagues and I have discussed implementing something similar to stale-if-error but have not yet gotten around to it. I look forward to trying out these features when Squid 2.7 becomes available.

As to whether the HTTP extension option or the cache configuration option eases maintenance the most in reverse proxy setups, I think it depends on local circumstances. But the HTTP extension option is the clear winner in performance terms, since it makes it possible for downstream caches (beyond one’s administrative capacity to configure) to participate in these schemes.

The Internet-Drafts say that these extensions MAY be implemented by shared caches and that private private caches MUST ignore them, but wouldn’t it be semantically equivalent for private caches to implement them also (while further boosting the performance benefits)? I suppose that doing so could heighten the security concern with stale-while-revalidate.

Also - the Internet-Drafts don’t say anything about cases where the RFC 2616 rules and these extensions come into conflict. For example, consider a request containing “Cache-Control: max-stale=600” that leads to a 500 error while refreshing a stale response (age=900) containing “Cache-Control: max-age=1, stale-if-error=1200”. Presumably the user agent gets the 500, but there is room for concern about correct implementation.

Alot of proxies out there already execute the stale-while-revalidate functionality as a “value-add”. Any content with explicit expire or max-age directives will continue to be served if caching of expired content is allowed, while the proxy asynchronously checks to see if there is a fresh object. For any objects with low or infrequent request volumes that change regularly, the user(s) caught in the middle get annoyed.

You can’t win for losing.

Another option might be “prefetch-while-stale”, although I can see that being rife with problems. This would allow a proxy to proactively refresh stale objects with this directive as soon as the object hits its expiration. A proxy admin could then choose to configure the proxy to enable this functionality with the default disabled.

Of course, I can see this directive used with a max-age of 0 or 1 or something silly like that.

Any thoughts on a stale-valid-until header that would indicate how long the 2 headers above should be in effect? I’m thinking about the occasionally connected case where I might want to have stale items used by my app until my device can reconnect but there are limits to that and I’d like the server to be able to indicate that with something like stale-valid-until.

To clariify my comment above, I’m wondernig about the use of stale content in an offline case and wondering whether that should fall under the revalidate case or if it should be given special consideration.