Hypothesis Driven Product Management

"The true method of knowledge is an experiment."

Sep 20, 2020

“The first principle is that you must not fool yourself and you are the easiest person to fool.”
- Richard P. Feynman

Lean hypothesis testing is an approach to agile product development that’s designed to minimize risk, increase the speed of development, and hone business outcomes by building and iterating on a minimum viable product (MVP).

The minimum viable product is a concept famously championed by Eric Ries as part of the lean startup methodology. At its core, the concept of the MVP is about creating a cycle of learning. Rather than devoting long development timelines to building a fully polished end product, teams working through lean product development build, in short, iterative cycles. Each cycle is devoted to shipping an MVP, defined as a product that’s built with the least amount of work possible for the purpose of testing and validating that product with users.

In lean hypothesis testing, the MVP itself can be framed as a hypothesis. A well-designed hypothesis breaks down an issue into a problem, solution, and result.

When defining a good hypothesis, start with a meaningful problem: an issue or pain-point that you’d like to solve for your users. Teams often use multiple qualitative and quantitative sources to the scope and describe this problem.

How do you get started?

Two core practices underlie lean:

Use of the scientific method and
Use of small batches. Science has brought us many wonderful things.

I personally prefer to expand the Build-Measure-Learn loop into the classic view of the scientific method because I find it’s more robust. You can see that process to the right, and we’ll step through the components in the balance of this section.

The use of small batches is critical. It gives you more shots at a successful outcome, particularly valuable when you’re in a high risk, high uncertainty environment.

A great example from Eric Ries’ book is the envelope folding experiment: If you had to stuff 100 envelopes with letters, how would you do it? Would you fold all the sheets of paper and then stuff the envelopes? Or would you fold one sheet of paper, stuff one envelope? It turns out that doing them one by one is vastly more efficient, and that’s just on an operational basis. If you don’t actually know if the envelopes will fit or whether anyone wants them (more analogous to a startup), you’re obviously much better off with the one-by-one approach.

So, how do you do it? In 6 simple (in principle) steps :
Start with a strong idea, one where you’ve gone out a done customer strong discovery which is packaged into testable personas and problem scenarios. If you’re familiar with design thinking, it’s very much about doing good work in this area.
Structure your idea(s) in a testable format (as hypotheses).
Figure out how you’ll prove or disprove these hypotheses with a minimum of time and effort.
Get focused on testing your hypotheses and collecting whatever metrics you’ll use to make a conclusion.
Conclude and decide; did you prove out this idea and is it time to throw more resources at it? Or do you need to reformulate and re-test?
Pivot or persevere; If you’re pivoting and revising, the key is to make sure you have a strong foundation in customer discovery so you can pivot in a smart way based on your understanding of the customer/user.