Cheap prototypes do not make product decisions cheap

Suppose your teams now use generative AI to build clickable prototypes in days instead of weeks. The sprint review fills with demos. Each one looks plausible, and each arrives with an unspoken question: should this go on the roadmap? As Head of Product or Head of Engineering, you have to decide whether to scale this way of working across more teams.

The demos are real. What remains unclear is whether anyone has learned that the underlying customer problem exists, that the proposed solution addresses it, or that the idea deserves a place in delivery. A prototype built in two days can still carry an assumption nobody has tested. If it moves into review, integration work, and roadmap commitments, the cost of that weak assumption has not gone down. It has only arrived sooner.

Cheaper code lowers the cost of producing an option. The cost of choosing the wrong option stays where it was, and the rate at which unexamined options reach the people who must evaluate them can go up.

What the research shows about AI and ideas

The research on generative AI in early innovation work is young, but it is fairly consistent on one bounded question: can AI produce good ideas quickly?

In several settings, it can. Joosten et al. (2024) compared packaging innovation ideas written by professionals with ideas produced by ChatGPT. An innovation executive at the company, who did not know the source of each idea, rated the AI ideas higher on novelty and customer benefit and rated feasibility as similar. Pescher and Tellis (2025), reviewing the wider literature, report credible gains in efficiency and in the average creativity of generated ideas.

So a blanket dismissal of AI ideation would misread the evidence. The more useful findings concern where those gains stop.

Where the gains stop

Eisenreich et al. (2024) ran an exploratory study inside a company working on sustainable packaging, comparing human, AI, and hybrid ideation conditions. AI produced highly novel ideas, while experienced employees produced more ideas that were highly feasible. Showing people AI-generated examples before they started could constrain their own later creativity. Employees also reported that expert input was still needed to turn generated options into proposals the organization could use. The study describes a trade-off between novelty and feasibility, with context supplied by people who know the constraints.

Doshi and Hauser (2024) add a second tension. In a preregistered randomized experiment, access to AI-generated story ideas improved the average creativity and quality of short stories, especially for writers who were less creative to begin with. The AI-assisted stories also became more similar to one another. Individual outputs improved while the collection as a whole lost diversity.

Pescher and Tellis (2025) point to a third limit. In their synthesis, AI performs much less well at selecting ideas than at screening them, and the evidence on whether it improves the very best ideas, as opposed to the average, is conflicting.

Taken together, these findings describe a pattern that matters for product work. AI raises the arrival rate of plausible options. Those options may lean toward novelty over feasibility, may converge on each other, and still need someone with context to judge which one deserves attention. Production gets cheaper; selection does not.

Why this matters for discovery and delivery

The argument has to stay careful here, because none of these studies tests the outcomes a Product or Engineering leader is accountable for. None follows an AI-assisted prototype from the choice of assumption, through customer evidence, to a roadmap or delivery decision. The research does not show that faster prototyping improves discovery.

What it supports is a mechanism. When generating options becomes cheap, everything around generation gains relative weight: the question a prototype should answer, whether alternatives stay genuinely different, what evidence would count against the favored idea, and who decides when to stop. If those elements stay informal, faster production can mean that weak assumptions reach review, integration, and roadmap commitments at a higher rate. Roadmap work in progress can grow, and decision latency with it, because more polished options compete for the same limited evaluation capacity. A count of prototypes would then show plenty of activity and say nothing about whether any assumption was rejected early.

A prototype learning contract

One practical response is to require a short learning contract before anyone builds a prototype. It fits on a card and takes minutes to write. Its job is to make the team state what the prototype should teach before the speed of building makes that question feel unnecessary.

The contract has five lines:

• Decision: Which product or delivery decision can this prototype change? If the answer is none, the prototype is a demo and should be labeled as one.

• Risky assumption: Which uncertain belief does it test? Choose the belief that would hurt most if it turned out false, rather than the one that is easiest to show.

• Disconfirming signal: What observation would make the team stop or revise the idea? Write it down before seeing results, so that enthusiasm for a polished interface cannot quietly replace the original question.

• Decision owner: Who makes the stop-or-continue call, and on which evidence?

• Decision date: By when? A date turns open-ended exploration into a decision with a visible cost of delay.

When teams use generative AI to create options, add one step before the contract. Generate several materially distinct alternatives before selecting one to prototype. Distinct alternatives rest on different assumptions about the customer problem or the solution; variations in layout or wording do not count. The convergence observed by Doshi and Hauser (2024) and the constraining effect of early AI examples in Eisenreich et al. (2024) suggest that the first plausible answer can anchor everything after it. Widening the set before narrowing it gives the selection step real choices and keeps the prototype from becoming a refined version of whatever the model offered first.

For a Head of Engineering, the contract also works as an intake rule. A prototype without a named decision, assumption, disconfirming signal, owner, and date has not yet earned integration work. That gives engineering a principled reason to keep exploratory output out of the delivery queue until someone has actually decided something.

What the evidence cannot tell you

The limits deserve a plain statement. The accepted studies cover bounded ideation tasks, expert ratings, one creative-writing experiment, and one review of an early, fast-moving literature. The two product-ideation studies each come from a single corporate packaging context, and one relied on a single evaluator. None measures customer learning, product outcomes, or delivery performance. The learning contract follows from the mechanism these studies describe; it has not itself been tested against those outcomes.

The decision in front of you

If you are deciding whether to scale AI-assisted prototyping, the research gives you reason to expect more options, often of decent average quality. It gives you no reason to expect better product decisions by default. Those still depend on choosing the right assumption, looking for evidence against it, and having someone accountable decide by a set date. Cheap prototypes make that discipline more important, because they remove much of the friction that used to slow weak ideas down.

References

Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces the collective diversity of novel content. Science Advances, 10(28), Article eadn5290. https://doi.org/10.1126/sciadv.adn5290

Eisenreich, A., Just, J., Gimenez-Jimenez, D., & Füller, J. (2024). Revolution or inflated expectations? Exploring the impact of generative AI on ideation in a practical sustainability context. Technovation, 138, Article 103123. https://doi.org/10.1016/j.technovation.2024.103123

Joosten, J., Bilgram, V., Hahn, A., & Totzek, D. (2024). Comparing the Ideation Quality of Humans With Generative Artificial Intelligence. IEEE Engineering Management Review, 52(2), 153–164. https://doi.org/10.1109/emr.2024.3353338

Pescher, C., & Tellis, G. J. (2025). The Role of Artificial Intelligence in the Ideation Process. Journal of Product Innovation Management, 42(5), 958–982. https://doi.org/10.1111/jpim.12791

AI disclosure: AI was used to help create this article. Responsibility for the published content remains with Nico Thümler.