Assumption Validation

Every assumption you make while building a product is a risk. Validation turns those assumptions into evidence, so you end up building something that solves a real problem and something target customers will actually pay for.

Gathering Evidence: is this problem real?

Before you build anything, get a deep understanding of the problem your target customer is really dealing with.

Anecdotal Evidence: what target customers say

Anecdotal evidence is a signal, not proof. It's useful for pointing you toward a problem worth digging into, but it's not the truth, you're still making assumptions at this point. Use anecdotal evidence to guide where you look next, not as the basis for what you build. A few ways to gather it:

  • - Customer Feedback: customer support tickets, customer reviews, online forums (customer communities), social media posts.
  • - Qualitative Surveys (Open-Ended): asking questions such as "what can we do to improve our product?", "what features do you expect from our product?" or "let us know about your experiences with our product."
  • - User Interviews: discuss in depth what a target customer thinks, likes, or dislikes.
  • - Focus Groups: discuss in depth what a group of target customers think, like, or dislike.
  • - Stakeholder Walkthroughs: discuss a project or show a prototype to key stakeholders to collect feedback.

Real Evidence: what target customers do

Real evidence shows a real problem exists. It's fine not to have this on day one, set up a research plan, and pull in a designer, user researcher, or engineer if you need help collecting it. A few ways to gather it:

  • - Analytics: adoption metrics, engagement metrics (session duration, bounce rate, conversion rate), retention metrics, rage metrics (rage clicks, typing in all caps), data mining (surfacing patterns), clickstreams, drop-off points (onboarding flow, form completion, cart abandonment), performance metrics (latency), click tracking, heatmaps, etc.
  • - Quantitative Surveys (Closed-Ended): ratings, purchase probability, stack ranking, trade offs, or asking questions such as "have you done X in the last week?".
  • - Screen Recordings: monitor how target customers walk through their task flow and where they fail.
  • - Field Studies, Contextual Inquiries, or Diary Studies: observe or record how target customers attempt to achieve their goals.
  • - Task Completion Testing (Moderated or Unmoderated): ask your target customer to complete a series of tasks, monitor where they fail.

Value Validation

Solving a real problem creates value, but how do you know if your solution creates enough value? How do you know if target customers would actually pay for this problem to be solved? The right test depends on how much you've already built:

No Prototype:

  • - Concept Testing: show a world where the problem is solved for your target customers, explain the simplest version of what you're planning to build, and surface if it meets the needs of your target customers.
  • - Willingness to Pay: learn if target customers are "willing to pay" for a solution to their problem before building it, get a ballpark idea of an acceptable price, learn if this price scales to meet your goals.
  • - MaxDiff Analysis: learn the most preferred and least preferred benefits your product could offer to target customers.
  • - Painted Door Test: create a "doorway" (link, button, etc.) that, when opened, notifies target customers you are working on solving this problem, measure how many target customers "open that door" to validate interest, recruit target customers for interviews or testing prototypes.
  • - Paid Waitlist: ask target customers to pay upfront for a solution that isn't built yet to validate demand.

Design-Only Prototype:

  • - Usability+Value Test: after usability testing, learn if the product solve the target customer problem: do they ask to sign up or pay, do they ask if they can use the product now, would they partner with you to test the product and provide feedback, will the target customer share the product on their social media right now?

Hybrid Prototype:

  • - Wizard of Oz Test: show a target customer a design prototype and manually "pull the strings" behind the curtain so it appears to function as if a production-ready product.

Live-Data Prototype:

  • - A/B Test: test the existing product against a prototype using 1%-50% of target customers (you do not need production-ready code), monitor key metrics to learn if the test group outperforms the control.
  • - Invite-Only Test: if you lack the users to reach the statistical significance of an A/B test, then invite a limited set of users to test the prototype.

Usability Validation

Your users need to be able to use what you build. Show a target customer a prototype and watch. Can they get through their task? Where does friction show up? End this interview with a value test.

Feasibility Validation

Your team needs to be able to build this, and you've got deadlines. Talk it through with your engineers, can they build it with the skillset on the team? What's their effort estimate?

Viability Validation

The product also needs to work for the business. Walk your stakeholders through the problem and the solution, and address concerns from sales, finance, legal, security, marketing, or whoever else needs to weigh in.