Outcomes: The Forgotten Test Case


Outcome versus Output

My very simple definition of the difference between output and outcome is:

  • Output is delivering volume
  • Outcome is delivering value

Theoretically, they could be the same but in practice this is seldom the case.

I can create a lot of ‘stuff’ (i.e. output) but realise very little, or even negative, value for all the time and effort spent producing said ‘stuff’. This article provides two practical examples of what normally happens (we focus only the output), why this is a terrible practice and the problems associated with ignoring the outcomes. Continue reading “Outcomes: The Forgotten Test Case”


Kohavi’s Law & Harry Potter explain why your experience and intuition suck


Ron Kohavi was the Technical Fellow and VP of the Analysis & Experimentation team at Microsoft. He had a team of brilliant engineers, data scientists and program managers working under him. Kohavi’s team built Microsoft’s “Experimentation Platform.”  Before the Experimentation Platform was in use, Microsoft teams were delivering well-thought out features, some small, others larger multi-month projects. Output was great – and Microsoft products like Bing, Edge, Exchange, Office, Skype, Windows, and Xbox were benefitting from the exceptional work produced.

Or were they?

Continue reading “Kohavi’s Law & Harry Potter explain why your experience and intuition suck”