don't worry, it's probably fine

Every change should alter a measure of success

01 Nov 2019

nablopomo measures

I think about why we do things quite a lot. In the shower, on the Tube, in meetings (sorry). One I’ve been wrestling with recently is “why do we refactor?”.

“But Alex”, I hear you say with incredulity, “it’s obvious! We want to make our code better.”

Okay, sure. But what’s “better”?

There’s been a lot said about numerical measures of success in a production-like environment - they take the form of KPIs/SLx and seem to be fairly well understood. Successful user journeys, conversion, error rates, and of course revenue.

I remember reading “every deploy should change a metric” somewhere but have been unable to attribute it. I feel this is good start but too focused on post-release performance, and I’d like to offer up the following, more general, alternative

Every change should alter a measure of success

I opted for “change”, as opposed to “deploy”, because there are some changes that will not make material difference to post-deployment metrics. Refactoring without changing functionality or performance is one of these things.

I opted for “measure of success” rather than “metric” because some of these measures will be subjective rather than a counter of things that will happen.

As an example, when we invert a negative if-else-statement into an early return, I believe that we are trying to change a measure of success even if we’re not able to nail it down precisely what that measure is.

It could be “levels of indentation”, or “I prefer this stylistically”, or even “I have a pathological hatred of if {} else {}”.

On a personal level, I largely attribute this to a low-level anxiousness I have about doing things that are at worst regressive and at best pointless.

I’m going to talk in a future post about an idea I have for a non-standard measure of success - average time since code last reviewed.

In summary:

  • Don’t get too wrapped up in production metrics as your only measure of whether things are going well.
  • Think about what’s important for success in your context, and think about how to measure it.
  • Talk to your team about what’s important to them.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 - some short and sweet, others long and boring.