What your signup numbers aren’t telling you

In preparing today’s webinar on early user discovery, I’ve been working with a fictional example: a product helping small agencies get work approved by their clients. Their problem? People sign up, but few get to the point where they’re actually receiving useful feedback. The founder wants to buy more traffic.

It’s an understandable move. More people coming in gives you more chances to turn someone into a customer. But it also leaves an important question unanswered: what’s really happening to the people who are already there? Spoiler alert: they can help you understand how to attract better-fit customers, rather than just more of them.

Maybe they’re looking for something the product doesn’t do. Maybe the website makes the right product sound like something else. Or maybe they understand the promise perfectly and get stuck trying to use it. Each explanation points to a different part of the customer experience. Your messaging sets the expectation; what happens next tells you whether the experience lives up to it. Looking at both helps you work out what to investigate first.

This is where research comes in, but it’s also where it can sound like another project standing between your team and the work it needs to ship. You need interviews, a survey, someone to analyze everything. Meanwhile, the onboarding emails still need writing and the campaign is supposed to go live.

Thing is, sometimes you do need that larger research behemoth of a project. But you don’t have to wait for it to start learning either. An interaction already happening in your business can give you a place to ask a more specific or targeted question, provided you’re clear about what the answer can and can’t tell you.

Start with the decision you’re trying to make

For our fictional founder, “understand our users” is too broad to help much. “Should we investigate the signup promise or the setup experience first?” gives the research a job.

Now an email to someone who signed up and stalled has a purpose beyond reminding them to come back. You could ask: “What was the first thing you were hoping to accomplish when you signed up?”

That question doesn’t ask them to design your onboarding. It asks about the task they brought with them. Their answer gives you something to compare with the journey your product actually offers.

The same approach works at other moments along the way. The trick is choosing the moment that fits the uncertainty you’re trying to resolve — and skipping the moments where you already have enough signal.

A useful question in a relevant email is a better starting point than a feedback prompt that appears simply because you can add one.

Put the answer beside what happened

Let’s stay with the agency tool. Imagine a new user replies that they wanted to send a design to a client for approval that afternoon. Your product data shows they started setting up a workspace but never sent a review link.

You now know more than “this person didn’t activate.” You have an intended outcome and a point where their progress stopped. But you still don’t know why.

Perhaps they couldn’t figure out the setup. Perhaps they understood it but didn’t have time. Perhaps the design wasn’t ready, or a colleague told them to use an existing tool instead. The reply and the activity record narrow the question but don’t supply the missing explanation.

A useful follow-up would be: “What happened when you tried to send the design?” If they describe getting stuck in setup, you could ask to watch them attempt that task. If they say they needed formal sign-off that the product doesn’t support, you’d investigate fit and expectations instead.

That’s the practical loop: ask about the intended outcome, look at what happened, consider more than one explanation, and choose the next check. Don’t turn the first plausible explanation into a redesign brief before you’ve checked it.

And if their answer matches their behavior, that’s useful too. It supports your understanding of that interaction.

Keep enough context for the next person to use it

The answer needs somewhere to go besides your inbox. Otherwise, the next person writing an email or reviewing the funnel has to start from the same assumptions you did.

You don’t need an elaborate repository to start. Keep the question, the reply in the customer’s words, the date, and the relevant behavior together in an appropriate team workspace. Separate what you observed from what you think it means. “Started setup, didn’t send a link” is an observation. “Setup is too complicated” is an interpretation you still need to investigate.

A simple tracking sheet is enough to start. Keep one row per interaction, with the source, question, customer words, date, observed behavior, interpretation, and next check. The last two fields keep your explanation separate from the evidence.

For our fictional example: the customer wanted a client approval; they started setup but never sent a review link. “Setup may be blocking them” is the interpretation. Watching them attempt the first review is the next check—not a confirmed fix.

That distinction becomes more important as you collect more replies. Similar language from different customers might point to a pattern. It might also come from asking the same leading question or hearing only from the people willing to reply. Include people who stalled or left where you can, not just the customers who are happy to talk to you.

If you use AI to group the responses, keep the original evidence accessible. A tidy summary can make several different situations look like one problem.

Let the evidence change the next move

The point isn’t to collect more customer language for its own sake. It’s to make a better decision about your work. Think like a scientist.

If people repeatedly expect an outcome the product doesn’t deliver, where does that expectation come from? That could involve the website, a sales conversation, or an assumption buyers bring from another tool. If they want an outcome the product does deliver but can’t reach it, watch them try to do it, or look at product analytics to learn what’s happening.

Those are different investigations. Their results might lead to a message test, a usability change, a clearer qualification step, or a larger research project.

What I don’t want is for “we need to do research” to become a reason to postpone every useful conversation until your team has more time.

Take one decision you’re making this week: find an existing customer interaction that could help you understand more about their experience or about a problem you’re having. Ask the right questions at the right time, save the answers, and check them against what you can observe. You may not have enough to change a campaign yet, but at least you can stop doing the wrong thing.

Discovery

From Guesswork to Signal

Today, 9 October, Christopher Richards and I are presenting the Early User Discovery Without a Research Team webinar. We cover how to collect early evidence, turn it into clearer positioning and messaging, and choose what to test next. We also look at where AI can help without treating simulated reactions as customer evidence. If you’re trying to make these decisions with a small team, the event details are on Luma. I’ve got 39 slides packed with useful and actionable frameworks for you.

Staying interested when the future is uncertain

Alberto Romero’s essay There Are No Interesting Things; There Are Only Interested People takes a wider view of curiosity and an unpredictable future. Alongside this week’s practical topic, it raises a useful question: are we still interested in what we might learn, or only looking for confirmation of what we’ve decided? Read the essay in The Algorithmic Bridge.

Resonance