This is part 2 in a 3-part series. Part 1 defined Pareto dominance and Pareto-efficient software. Part 3 is a case study.
I have some advice that now feels (to me, subjectively) too obvious to bother to tell you. But it was once pretty contentious here in the beehive. So I predict you’ll either find this obvious or find it obviously wrong. Are you ready?
The Pareto Dominance Principle (PDP) requires that, when changing the functionality of your product, you can never accept even a small degradation for any (reasonable) user. The extreme of this is spacebar heating:
But the principled exception is that you can violate Pareto dominance as long as you write a blog post about it. Like when we killed freebee goals or killed exponential roads.
The argument against the PDP is that any change that’s better on net is worth deploying. Constant, continuous improvement! And also that you might find yourself at a local maximum, where you have to take a step back to take two steps forward.
I think those arguments are wrong. Deploying a change that’s a net improvement but not a Pareto improvement tends to mean tolerating sloppiness, and that can be a slippery slope. The PDP is a kind of commitment device against sloppiness. As for the local maxima argument, it’s so rare in practice that if it does happen, it’s worth blogging about.
I find that if you keep jumping on seemingly small Pareto improvements it ends up leading to unexpectedly drastic improvements. Imagine chipping away at bits of cruft here and there and occasionally avalanches of cruft fall away. Like a user-visible analog of constant opportunistic code refactoring. Without that discipline, it can be the opposite: new features or improvements having cascades of unintended consequences, agonizing tradeoffs, and technical debt creation.
In short, if you want to make a change that degrades even an obscure use case, or would make a single (reasonable) user even slightly less happy, maintain the discipline of proving that the change is fully intentional by spelling out and justifying the tradeoff. (Conversely, probably don’t add features for obscure use cases or to make single users happy!)
We commonly think in terms of User-Visible Improvements (UVIs) here at the beehive so here’s perhaps the pithiest possible version of the Pareto Dominance Principle: Negative UVIs verboten! Or less esoterically: Zero tolerance on regressions!
We haven’t always followed this religiously in the past but now that we’ve written this blog post, we’re on the hook.
Caveats and Fine Print
Thanks to the daily beemail subscribers who vetted this! Here are some exceptions and clarifications you all helped us come up with:
Conceivably we’ll someday need to make a Pareto dominated security fix. And less conceivably but still conceivably it would be something that’s a bad idea to discuss publicly. Typically it’s good to blog about the security fix after it’s fixed. But just in case, we’ll stake out security fixes as an exception.
Hopefully this isn’t too weaselly but we have the “reasonable user” clause as a kind of get-out-of-jail-free card. For example, if we didn’t have the previous caveat and had to do something for security reasons then a reasonable user can’t reasonably object. Or imagine a blatant bug that turns out to benefit some user in some way (like the spacebar heating in the xkcd comic). Being a blatant bug, a reasonable user can’t object to fixing it. They’d just have to make a feature request for addressing their use case in a non-buggy way!
The PDP is a constraint on what we knowingly deploy. We accidentally introduce bugs all(llll) the time and that doesn’t violate the PDP. Fixing regressions is always higher priority but the PDP doesn’t impose any particular commitment there either.
Thanks in particular to Jesper Dramsch, David MacIver, Dollar Flipper, and zedmango for helping formulate the fine print on this.