Every Feature Needs a Hypothesis


In this article, I’ll use a simple, easily accessible example feature to show how a feature hypothesis should evolve and how monitoring results can impact our plan and backlog priorities.

I am going to stick with the “Password Reset” example from the previous article in this “Outcomes versus Outputs” series since resetting one’s own password can be applied to almost any website and application. Continue reading “Every Feature Needs a Hypothesis”


Money or the Goody Bag? The Great Comrades Refund Debate

[This article was originally published on Sport24.co.za]

At lunch time on 14 May, I was taking advantage of a gap between video calls by doing another lockdown run up and down my driveway.

As you can imagine, driveway laps are not particularly interesting and I welcome any distraction from the monotony. Therefore, every vibration on my phone results in an excited glance at the screen to see whatever notification has been delivered.

I was about 30 minutes into my regular 10km jog when the latest buzz presented the email notification, “Media Release: 2020 Comrades Marathon has been cancelled.

I didn’t even break stride. Continue reading “Money or the Goody Bag? The Great Comrades Refund Debate”


Outcomes: The Forgotten Test Case


Outcome versus Output

My very simple definition of the difference between output and outcome is:

  • Output is delivering volume
  • Outcome is delivering value

Theoretically, they could be the same but in practice this is seldom the case.

I can create a lot of ‘stuff’ (i.e. output) but realise very little, or even negative, value for all the time and effort spent producing said ‘stuff’. This article provides two practical examples of what normally happens (we focus only the output), why this is a terrible practice and the problems associated with ignoring the outcomes. Continue reading “Outcomes: The Forgotten Test Case”


Kohavi’s Law & Harry Potter explain why your experience and intuition suck


Ron Kohavi was the Technical Fellow and VP of the Analysis & Experimentation team at Microsoft. He had a team of brilliant engineers, data scientists and program managers working under him. Kohavi’s team built Microsoft’s “Experimentation Platform.”  Before the Experimentation Platform was in use, Microsoft teams were delivering well-thought out features, some small, others larger multi-month projects. Output was great – and Microsoft products like Bing, Edge, Exchange, Office, Skype, Windows, and Xbox were benefitting from the exceptional work produced.

Or were they?

Continue reading “Kohavi’s Law & Harry Potter explain why your experience and intuition suck”


Bonkolo Marathon (Warming up for a World Cup win)

[MARATHON #230 / Unique Marathon #131 / 2 November 2019]

Where were you when the Springboks won the Rugby World Cup for the third time?
In an ethnic hair salon in Stutterheim is my unlikely reply.
How I got there was a result of bad planning and incredibly low expectations.

With the Rugby World Cup final a couple of hours away, the support tables during the Bonkolo Marathon were in full swing. I asked these guys who was going to win. They answered correctly: ‘Bokke!’

South Africa has three marathons on the first weekend of November – Soweto, Kaapsehoop and Bonkolo. Soweto and Kaapsehoop are two of the biggest marathons in South Africa whereas Bonkolo, with just 120 runners, is definitely the acutest angle on this running triangle. Continue reading “Bonkolo Marathon (Warming up for a World Cup win)”