Browser testing

On just about every client project that I work on, the subject of browser support comes up. Rightly so. It’s an important issue on which to get mutual understanding and agreement. But all too often, this important question is framed in a binary, true/false, go/no-go way: “Which browsers do we/don’t we support?”

Really, the first thing to get agreement on is not a list of browsers, but what we mean by the word “support”. In my mind, that word implies that a user of a particular browser should be able to accomplish the primary tasks on the website, whether that’s reading an article, booking a ticket, or buying a product. That doesn’t mean that the task must be experienced in pixel-perfect fidelity to an ideal visual design.

But to others, that’s exactly what “support” means. Personally, I’d call that optimisation. As Brad puts it:

There is a difference between support and optimization.

So to put it in glib terms, I support every browser …but I optimise for none.

Alright, fine. But I still need to get to some mutual understanding with a client about which browsers will get the optimised experience and which browsers will simply be supported.

Personally, I like the Filament Group’s approach of discussing this in terms of features rather than browsers. It makes sense to me to say the browsers that support geolocation will get the geolocation features, or the browsers that support offline caching will get the offline caching features. There’s no need to produce a list of what those browsers are for each feature, and in any case, the list would be constantly changing and updating with each new browser release.

But—and this is a big but—nine times out of ten, when the issue of browser support comes up, it isn’t about functionality; it’s about branding. What clients generally want to know is which browsers will get the ideal visual design. Obviously the newer versions of Chrome and Firefox are going to get all the lovely layouts, rounded corners, gradients, transparencies, and animations …but what about older versions of Internet Explorer? Even if users of IE8 and IE7 can accomplish their tasks, will the “degraded” visual presentation hurt their experience?

My hypothesis is that it won’t. Users of older versions of Internet Explorer aren’t doing a side-by-side comparison of the same website opened up in the latest Chrome nightly. Considering what their daily usage must be like—unable to use Facebook, unable to use Google services—I suspect that they are happy just to be able to complete their task, regardless of the site’s visual fidelity.

There’s another viewpoint—one that I’ve heard expressed by clients—that even users of older browsers should still get the ideal, pixel-perfect visual design. The hypothesis here is that, by allowing someone to experience anything less than the perfect presentation, the client’s brand will be damaged in the mind of that person.

Like I said, this is something that comes up on most client projects, and this is the point at which we’d have to come to an agreement about which hypothesis we’re going to go with. Of course I’m going to argue in favour of the first hypothesis, but I’ve come to realise that arguing in favour of either hypothesis is the wrong approach. We shouldn’t be debating this …we should be testing it.

We have two competing hypotheses about a group of users. Instead of trying to read their minds, why not test with that group of users to find out which hypothesis is correct? No matter what the results of the test, they will be valuable either way.

Think about the amount of work that’s going to go in to optimising for older browser versions—it’s going to take quite a bit of time and money. It makes sense to ensure that this time and money isn’t being spent on little more than a hunch that pixel-perfection is important to those users. On the other hand, if the test reveals that actually those users really will have a lesser opinion of a brand unless they get pixel-perfect parity with newer browsers, then you’ll know that the time and money spent making that happen isn’t wasted.

Josh wrote recently that 1 hour of research saves 10 hours of development time:

Or, in longer terms if more people appreciated how one day of user research can save weeks of coding I think they would do it more. It is remarkable what you decide to not build after talking to a few people closely.

When it comes to decisions around browser support/optimisation, I think that even a little bit of up-front research and testing could potentially save a lot of time, money, and heartache. I’m not sure exactly what form the testing should take, but I’m interested in figuring it out.