When Numbers Lie: A UX Designer’s Wake-Up Call from Metrics

You sit at your desk, reviewing the onboarding flow you just redesigned for the mobile app. Weeks of work went into this: consolidating unnecessary steps, clarifying confusing fields, and rewriting instructions that had caused users to get stuck in the past. The flow is smoother now, and most of the obvious pain points from earlier testing are gone.

Your team has been watching metrics closely. The product manager and engineers are eager to see results. Now that the redesign is live, you feel cautiously optimistic.

At first glance, the numbers look promising. Click-throughs are high. Users spend slightly more time on tutorial screens. Engagement seems up. It feels like the work paid off.

But during the next team meeting, the product manager frowns.

“Completion rates haven’t improved,” they say. “People are still dropping off mid-flow.”

Your stomach knots. How can users be clicking through at such a high rate if they’re not finishing? You assumed the redesign addressed the major friction points, but now it seems something unexpected is still causing confusion.

Over the next few days, you dive deeper. You watch remote sessions, review recordings, and — importantly — run a few real-time testing sessions in the office, inviting users to walk through the flow while you observe. You notice hesitation, repeated clicks, and moments where users pause, unsure what to do next.

Patterns emerge. One optional question about notifications is confusing — users don’t know if it’s required, so some skip it, and others abandon the flow. A subtle wording choice in a confirmation screen leaves a few unsure if they’ve completed the step correctly. A couple of users repeatedly tap “Next,” trying to move forward faster, which inflates the click metrics.

The analytics had made it look like the redesign worked, but live testing shows a different story: users are struggling in ways numbers alone can’t show.

Digging into completion times gives another clue. The averages suggest flows are moving faster than before, but segmentation reveals a hidden truth. About 20% of users take three times longer than average. Many abandon mid-flow. Most are on older devices, where slower load times amplify the friction. The average masked real problems that frustrated users.

Armed with these insights, you start iterating. The optional notification question is clarified, confirmation screens get stronger visual cues, and small wording tweaks reduce ambiguity. You retest, both live and remotely. This time, completion rates climb. Drop-offs decrease. Users move through the flow confidently, without repeated taps or hesitation.

You lean back and exhale. The metrics now align with what you observed in real time. Numbers are powerful, but they can mislead if you don’t combine them with observation, testing, and context.

By the time you close your laptop, the flow works as it should — but more importantly, you know that real UX is about understanding users, not just chasing numbers.

Helping UX Designers bridge gaps and grow