Minding our language

Considering the intentionality of our words

2 min read

I often hear statements such as “My hypothesis is that this page isn't performing well because the button is too far down" (or swap this reason for any other guess). This fictitious example may well be proved true after being tested, however, it is not a hypothesis.

While poking at semantics may seem nitpicky, I nevertheless believe it's helpful to remember that an assumption is not a hypothesis, and a hypothesis is not an assumption.

An assumption is just that — an assumption. In other words, an un-evidenced opinion. The great thing about stating assumptions is that we don't need any supporting evidence. We don't need a workable way to test the assumption, nor do we need success criteria.

A hypothesis, however, needs to be accompanied by a prediction of what will change and how we'll know if our prediction was accurate. It helps us formulate the assessment criteria for our assumption. In other words, it's a plan.

Not all assumptions are created equal

A good way to check if your assumption 'has legs' is to mould it into a hypothesis, because a hypothesis requires evidence. This in turn helps us determine what to do with our assumption.

Whilst there are many different ways of phrasing and testing hypotheses, in the context of planning A/ B testing I find any template that requires evidence to be useful, such as the following:

Because we have seen that... [evidence]
We believe that... [our assumption / the change we will make]
Will... [how it helps customers]
Which will... [observable change in business metric]

We need to validate test our hypotheses

It's also important to acknowledge our biases and consider how our language influences our approach. Consider the nuance between the two statements below:

“Next sprint, we need to validate our hypothesis” VS “Next sprint, we need to test our hypothesis”

In the first statement, we're telling on ourselves. We plan to find information to validate that we're correct. After all, it needs to be done next sprint — we don't have time to be wrong! This encourages confirmation bias.

The second statement encourages us to keep an open mind. We're allowing ourselves to take a neutral, curious approach to learn how strongly our assumption may be supported or refuted. The implied outcome is learning.

Great! We've validated supported our assumption

'Validated' suggests we are right. There was one right answer, and we found it. Go team! A subtle shift from 'validated' to 'supported' carries less hubris. This acknowledges we're going in the right direction. But maybe there are other ways we can further support our assumptions too.