Why customers buy, switch and leave: a map of the research methods

Three questions that most research does not answer

Most product teams do research. Few of them can answer three questions about their own customers: 1) Why did this customer buy? 2) Why did this customer switch to us from something else? 3) Why did this customer leave?

The three questions look simple but they are not: a customer does not buy a product. A customer hires a product to make progress in a situation. Without knowing the context, you cannot sell your product (or service). The situation comes first, the product comes second. Most research methods start with the product. So most research methods miss the situation.

This text is a map that shows the common research methods that a product team uses today. For each method, the map states one thing: which question the method answers well, and which question the method cannot answer. Then the map shows where Jobs to be Done sits. Jobs to be Done is a way of thinking, and an interview method, that starts with the situation and not with the product.

The map does not teach you how to run each method. A map shows where the roads go but it does not drive the car. If you want to learn one method in depth, this text tells you which method, and why. The learning itself is the next step, and that step is yours.

One warning: every method below is good at something, none of them are bad. Don't make the mistake of asking a method a question that it cannot answer and then to still rely on the answer.

The map: what each method answers, and what it cannot answer

The methods below fall in three groups. The first group watches what people do. The second group asks people what they think. The third group tests whether people can use a thing. The three groups answer different questions. No group answers all questions.

Group 1: methods that watch behaviour

Product analytics. Product analytics is the count of what users do inside your product: which pages they open, which buttons they click, how often they return. Analytics answers one question well: what happens inside the product, and how often. Analytics cannot tell you why a user did something. Analytics also cannot see anything that happens outside the product. The decision to buy happens outside the product. The decision to leave usually happens outside the product too. Analytics records the footprint but it doesn't record the walk.

Session recordings and heatmaps. A session recording is a video of one user's screen while that user works in your product. A heatmap is a picture of where many users click or scroll on one page. Both methods answer this question well: where in the product do users hesitate, stop, or go around an obstacle. Both methods cannot tell you what the user wanted to achieve, or what the user did before they opened the product.

A/B tests. An A/B test shows version A of a screen to one half of the users and version B to the other half, then measures which version produces more of a chosen action. An A/B test answers one question well: which of two versions produces more of that action. An A/B test cannot tell you which versions to build in the first place. An A/B test also cannot tell you why version B won. A/B tests also require you have a lot of active users, otherwise the results are not relevant. You measure the winner without learning the reason.

Support tickets and cancellation forms. A support ticket is a message from a customer with a problem, and a cancellation form is the short form a customer fills in when they leave. Both answer one question well: what breaks, and what a leaving customer says on the day they leave. Neither tells you why the customer really left: on the cancellation form, most customers pick the reason that is quickest to click, and the real reason was usually decided weeks earlier.

Group 2: methods that ask people

Surveys. A survey is a fixed list of questions sent to many people. It answers one question well: how many people, out of a known group, pick each answer to a question you already know how to ask. A survey cannot find a question you did not know to ask, and it cannot explain a single answer. If 40 percent say price was the reason, the survey doesn't tell you what price was compared against.

Net Promoter Score. Net Promoter Score (NPS) is one survey question: how likely are you to recommend this product to a colleague, on a scale from 0 to 10. NPS answers one question well: how the mood of your customer base moves over time. It cannot tell you what moved the mood, and it cannot predict whether one specific customer will stay or leave (a 9 has left, a 6 has renewed for four years). The question itself has a deeper flaw: it asks people to predict their own future behaviour, and people are bad at that, so the number is mostly noise dressed as a trend. Jared Spool makes the full case in Net Promoter Score Considered Harmful; read it once and you will not defend NPS again.

Customer satisfaction score. A customer satisfaction score (CSAT) is one question asked right after one interaction, such as a closed support ticket or a purchase: how satisfied were you, on a scale from 1 to 5. CSAT answers one question well: how one specific interaction landed, while the customer still remembers it. It cannot tell you how the relationship stands as a whole, and it cannot tell you whether the customer will stay. A customer can rate every support ticket a 5 and cancel next month because the product no longer does the job.

Feedback calls and check-ins. A feedback call is a conversation, usually led by a customer success manager or an account manager, in which the customer says what they like and what they miss. It answers one question well: what the customer wants from you next, in their own words. It cannot tell you why the customer bought: the customer talks about the product they have today, and the moment they decided to go looking for one is months behind them.

Focus groups. A focus group is a discussion with 6 to 10 people at once, led by a moderator. It answers one question well: how people talk about a topic in front of other people, and which opinions travel fast in a group. It cannot tell you what one person did alone, in their own situation, with their own budget. In a group the loudest opinion wins, and nobody makes a purchase decision in a group.

Customer discovery interviews. A customer discovery interview is a one-to-one conversation, usually early in a product's life, in which a product person asks about the problems a customer has. The best known version comes from the Lean Startup movement and from the book The Mom Test, which teaches you to ask about the past and never about the future. A discovery interview answers one question well: which problems exist, and how painful each one is today. It cannot tell you what makes a person act on a problem. Many people carry a painful problem for years and buy nothing, so finding the problem is only half the story; the trigger is the other half.

Win-loss interviews. A win-loss interview is a conversation with a buyer after a sales cycle ends, to learn why they chose you or someone else. It answers one question well: which arguments counted in the final comparison between you and a competitor. It cannot tell you what started the search, because the interview begins at the shortlist and the decision to search began long before that.

Group 3: methods that test whether people can use a thing

Usability tests. A usability test puts one person in front of your product, gives them a task, and watches whether they can complete it. It answers one question well: can a person use this screen to do this task, and where do they get stuck. It cannot tell you whether the person wants to do that task at all. A screen can be easy to use and still solve nothing.

System Usability Scale. The System Usability Scale (SUS) is a fixed set of ten statements a person scores after using a product; the scores add up to one number from 0 to 100, and 68 is the average across products. SUS answers one question well: how usable the product feels to its users, in a form you can compare across releases and against other products. It cannot tell you where the usability problems are, and it cannot tell you whether the product does a job anyone needs done. A product can score 85 on SUS and still be something nobody would pay for.

Prototype tests and concept tests. A prototype test shows a person a mock-up of a product that doesn't exist yet and asks them to react; a concept test does the same with a written description of an idea. Both answer one question well: does the person understand what this thing is, and can they say what they would use it for. Neither tells you whether the person will buy it. A reaction to a picture is not a purchase, and people are polite to pictures.

Card sorting and tree testing. Card sorting asks a person to group labels into categories, and tree testing asks a person to find an item in a menu structure. Both answer one question well: which words and which menu structure match the way a person already thinks. Neither tells you what the person is trying to achieve when they open the menu.

The map on one page

The table below repeats the map in short form. Each row is one method, the second column is the question the method answers well, and the third column is the question the method cannot answer.

Read the third column from top to bottom: every method on the map fails on the same family of questions. Why did the customer start looking? What did they compare, including the option to do nothing? What made them decide on one day and not another? Those are the questions of buying, switching and leaving, and none of these methods was built for them.

Where Jobs to be Done sits on the map

Jobs to be Done is the method built for the third column of the map: why did the customer start looking, what did they compare, and what made them decide. It has one core idea and three tools.

The core idea: people hire products to make progress

A customer doesn't want a product. A customer wants to move from the situation they have to a situation they want, and the product is the thing they hire to make that move. That move is the job. The phrase Jobs to be Done means exactly this: the progress a person is trying to make in a specific situation.

This changes the research question. If the job comes first, the question is no longer "what do you think of our product?" but "what happened in your life that made you go looking for something?" The first question is about the product, the second is about the situation, and the situation is where buying, switching and leaving are decided.

It also changes who your competitor is. If people hire products to make progress, your competitor is anything else a person could hire for the same progress: a spreadsheet, a colleague, or doing nothing at all. Most product teams list three competitors. The customer's list is longer, and the customer's list is the one that counts.

Tool 1: the switch interview

The switch interview is a one-to-one conversation with a person who recently bought, switched or cancelled. It lasts 45 to 60 minutes and has one subject: the story of that one decision, from the first moment the person thought about it to the day they acted. The interviewer doesn't ask for opinions but for the story, in order, with dates, places and the people who were in the room.

The difference with a discovery interview is this: a discovery interview asks about problems in general, a switch interview asks about one purchase that actually happened. Because it happened, the person remembers the details, and because they remember the details, the interviewer can find the moment the decision was made. A discovery interview never reaches that moment.

Tool 2: the timeline

The timeline is the shape every purchase story takes when told in full. It starts with the first thought: the moment a person notices that the current way isn't good enough. Then comes passive looking, where the person notices options but doesn't act. Then active looking, where the person sets aside time and compares options. Then deciding: the person picks one and commits. After the purchase come two more stages, first use and the moment the person either forms a habit or gives up.

The interviewer uses the timeline as a checklist. If the story has no first thought, you haven't gone back far enough. If it jumps from passive looking to deciding, something pushed the person over the line and you haven't found it yet. The timeline turns a story into a set of events you can compare across people.

The replay: a recording of the journey, taken from people who completed it

Put the switch interview and the timeline together and you get something no other method on the map gives you: a recording of the buyer's journey, made with the people who ran through it and came out the other side as customers. Each interview is one recording. Ten interviews, laid on the same timeline, are the route.

That recording has two uses. The first is replay. A person who is new in the journey, somewhere between the first thought and deciding, is walking a route others have walked before them. You can play the recording back to them: this is what the people before you could not stand any more, this is what they hoped for, this is what they feared, and this is what pushed them over the line. A pilot on an unfamiliar approach follows a chart drawn from the pilots who flew it before; the chart doesn't fly the plane, but it tells the pilot what comes next.

The second use is position. Because the timeline has fixed stages, you can tell which stage a new person is in from what they say and do. Someone who complains but hasn't looked at options is at the first thought. Someone who has a shortlist and a date is deciding. Every stage asks for a different conversation, and the recording tells you which one. Analytics can't place a person on that route, because the route starts long before anyone opens your product. A survey can't either, because it doesn't know the stages exist.

This is the one thing the map was missing: a method that not only explains past decisions, but lets you recognise where a new customer stands in theirs, and speak to that stage.

Tool 3: the four forces

The four forces explain why a person moves along the timeline, or stops. Two push toward change: the push of the current situation (what the person can't stand about today) and the pull of the new solution (what they hope the new option gives them). Two push against change: the anxiety about the new solution (the fear it will fail or cost more than expected) and the habit of the present (the comfort of the known way, even a bad one).

A person switches when push plus pull beat anxiety plus habit, and stays when anxiety and habit win. This is why a painful problem alone doesn't cause a purchase. Discovery interviews find the push, which is one force out of four. The other three stay unmeasured, and any of them can block the sale.

The same four forces explain leaving. A customer leaves when the push of your product (what they can't stand about it any more) plus the pull of an alternative beat the habit of staying with you and the anxiety of moving. A cancellation form records the push only; a switch interview with a leaver finds the other three.

The three questions, answered

With the switch interview, the timeline and the four forces, the three questions from the start of this text have a method behind them. Why did the customer buy: the story from first thought to deciding, with the push and the pull named. Why did they switch to us: the same story, with the habit and the anxiety of the old option named, and the moment those two were beaten. Why did they leave: the same story again, with your product now on the losing side of the four forces.

What Jobs to be Done adds, and what it cannot do

Jobs to be Done doesn't replace the methods on the map. It fills the third column and leaves the second column alone.

What it adds to the other methods

Analytics shows that users drop off at step 3 of onboarding; a switch interview tells you what the user hired the product to do, so you can judge whether step 3 serves that job or blocks it. Analytics gives the where, the interview gives the why.

A survey shows that 40 percent of leavers name price. A switch interview with five of those leavers tells you what price was compared against: a cheaper tool, a colleague's time, or doing the work by hand. The survey counts; the interview explains what is being counted.

A discovery interview finds a painful problem. A switch interview tells you whether people with that problem actually act on it, and what pushes them to. The discovery interview finds the push, the switch interview finds the other three forces.

A win-loss interview tells you which argument won at the shortlist. A switch interview tells you how the buyer got to a shortlist at all, and what they would have done without one. The win-loss interview starts at the end of the story, the switch interview at the beginning.

A usability test tells you a screen is easy. A switch interview tells you whether anyone hired the product to reach that screen. Easy is only good when the task matters.

Every method on the map looks at one moment: a click, a score, a ticket, a shortlist. A set of switch interviews on one timeline gives you the whole route, recorded with people who completed it. That recording lets you replay the journey for a new customer and tell which stage they are in, which is something no click, score or ticket can do on its own.

What it cannot do

Jobs to be Done gives depth, not width. One switch interview gives one person's full story, ten interviews give a pattern, and ten interviews still don't give a percentage. If you need to know how many customers share a pattern, a survey built on the interview findings gives the number.

It cannot tell you whether a screen works. The four forces say nothing about button placement; a usability test does. Both questions are real, and they need different methods.

It cannot tell you what will happen. A switch interview is a story of a decision that already happened, so on its own it predicts nothing. It gives you the pattern other decisions are likely to follow, and a test (an A/B test, a small launch) shows whether the pattern holds.

It depends on the interviewer. A switch interview run as a discussion guide, with a fixed list of questions and no follow-up on the story, produces the same confirmation research as any other method. The method doesn't protect you from a bad interview. Only practice does, and practice is only visible when someone reads the interview back and tells you where the story was lost.

The map with Jobs to be Done added

The last four rows were empty before; the other rows are unchanged.

Where to start

You don't need to learn every method on the map. You need to know which question you have, and then pick the method that answers it. The steps below are for a person whose question is one of the three: why do customers buy, switch or leave.

1. Write down which of the three questions you have, and pick one. "Why do customers leave" is one question. "What do customers think of us" is not on the map, and no method answers it.
2. Find five people who made that decision in the last 90 days: five recent buyers, five recent switchers, or five recent leavers. Recent matters more than many. A person who left last month remembers the story; a person who left last year remembers a summary.
3. Run five switch interviews. Ask for the story from the first thought to the day they acted. Don't ask what they think of your product, ask what happened.
4. After each interview, place the story on the timeline (first thought, passive looking, active looking, deciding) and mark the four forces you heard: push, pull, anxiety, habit. Also mark the ones you didn't hear. Those are gaps in your interview, not gaps in the customer's story.
5. After five interviews, look for what repeats. The repeats are your first pattern. Only then decide whether a survey should count how many customers share it.

The five steps take about three weeks. The first interview will be bad and the fifth will be better, and the difference between the two is practice.