Synthetic Users: How to Run AI-Simulated Customer Interviews (and When Not To)
The promise of AI-simulated customer research is seductive: instead of spending two weeks recruiting, scheduling, and interviewing twelve users, ask an AI to respond as each of them would. Instant feedback. Unlimited iterations. Zero scheduling overhead. Teams that have tried synthetic user interviews for the first time often come away impressed — the simulated responses are coherent, nuanced, and faster than anything a real recruiting process could produce. The question is not whether the outputs are impressive. The question is whether they are true.
AI language models generate responses that are statistically consistent with patterns in their training data. When you prompt a model to respond as a 35-year-old product manager with two years of experience using project management software, it generates a response that is statistically representative of how people matching that description have written about project management in the data the model was trained on. This is useful information. It is not the same as speaking with an actual product manager about your specific product, in the context of their specific workflow, at the moment they are experiencing the specific problem you are trying to solve. Distinguishing between these two kinds of information — and understanding when each is appropriate — is the critical capability for designers using AI simulation in product research.
Synthetic users work best as a preparation tool for real research, not a replacement for it.
Where Synthetic Users Are Genuinely Useful
Synthetic user simulation is most valuable in two specific research contexts. The first is early-stage assumption stress-testing, before any real user contact is possible. When a design team is evaluating three possible approaches to a user interface problem and needs to understand which approaches are likely to surface comprehension issues, emotional friction, or workflow conflicts, simulating a range of user responses can identify which approaches warrant real-user testing and which can be eliminated from consideration. This use of simulation saves real-user research budget for higher-value decisions by using AI to filter the option space before investing in empirical testing.
The second valuable context is edge-case coverage. Real-user research is expensive, which means it tends to be conducted with small samples that may not include the full range of user contexts the product will encounter. AI simulation can generate responses from users representing demographic contexts, technical environments, or use case variations that a twelve-person interview study would not cover. A design team testing a new onboarding flow can use simulation to generate responses from users with accessibility needs, from international users navigating the product in a second language, or from users who are encountering the product in low-bandwidth environments — providing coverage that direct research would require a much larger study to achieve.
The unexpected behavioral insights that most change product direction can only come from real user observation.
The Critical Limits of Synthetic Research
Synthetic user research fails predictably in two situations that are, unfortunately, the most important situations for product design. The first is novel product contexts. Language models generate responses based on patterns in existing data. When the product you are building does not resemble anything in the model's training data — because it addresses a problem in a genuinely new way, or because it targets a user population whose experiences are underrepresented in the training corpus — the model's simulated responses will be extrapolations from adjacent contexts that may not transfer accurately. Designing based on synthetic responses to a novel product is equivalent to designing based on research about a different product.
The second failure mode is the discovery of unexpected user behavior. The most valuable insights in product discovery are the ones you did not anticipate — the user behavior patterns that contradict your assumptions and reveal a fundamentally different mental model of the problem. These unexpected insights are precisely what AI simulation cannot produce, because the model generates responses consistent with patterns in its training data, which means it will confirm whatever the data confirms and cannot surface the genuinely novel behavioral pattern that only emerges from observing a real user in their actual context. A synthetic user will not have the specific, idiosyncratic workflow workaround that a real user has developed over three years of using the product's predecessor. The absence of that specificity is not a gap you can compensate for with a larger simulation sample.
Integrating Synthetic and Real Research Responsibly
The productive integration of synthetic and real user research treats simulation as a preparation tool for real research, not a replacement for it. Use simulation early to generate the questions you most need to answer with real users: which of the assumptions your design makes are most uncertain, which behavioral predictions are most worth testing, which user segments are most likely to have experiences your team has not considered? These questions become the agenda for real user interviews.
After real research, use simulation to extend coverage: you have interviewed twelve users and have high-quality insights from those twelve people. Simulation can help you evaluate whether those insights are likely to generalize to user segments you did not interview, flagging cases where the simulated responses diverge significantly from your real-user findings as potential places where additional real research is warranted. This hybrid approach — simulation for coverage, real research for validity — uses each tool for what it does best. It also maintains the safeguard that Lean UX requires: actual behavioral measurement with real users as the foundation of product decisions, with simulation providing useful supplementary input rather than substituting for the empirical foundation.
The Bottom Line
Synthetic users are a powerful tool in the right hands for the right purposes. They accelerate the early stages of discovery, extend the coverage of real research, and provide a cheap filter for design options that would otherwise require expensive real-user testing to eliminate. They are dangerous when mistaken for a substitute for real user contact, when used to validate novel product concepts that have no analog in training data, or when treated as a source of unexpected insights that can replace the genuine surprise of direct observation. Designers who use AI simulation with clear-eyed understanding of its limits will find it a valuable accelerator. Those who use it without those guardrails will find it a confident source of systematically wrong answers.
Related Posts from Sense & Respond Learning
AI as a Discovery Engine: Using Language Models to Accelerate Assumption Testing
Proto-Personas: How to Create User Alignments in Under an Hour
Integrating User Research into Every Sprint: The 3-12-1 Method
The Personalization Trap: Why More AI Data Doesn't Automatically Produce Better Products
Further Reading & External Resources
Lean UX — Gothelf & Seiden (O'Reilly) — The behavioral outcome framework that real research must ultimately validate
Just Enough Research — Erika Hall — Practical guide to choosing the right research method for the right question
The Mom Test — Rob Fitzpatrick — Why real customer conversations remain irreplaceable in product discovery
Want to go deeper? This post is part of the Sense & Respond Learning resource library — practical frameworks for product managers, transformation leads and executives who want to lead with outcomes, not outputs.
Explore the full library at https://www.senseandrespond.co/blog
