Better Models, Worse Tools: Why AI Progress Is Breaking Product Design

TL;DR

• AI model improvements create tool complexity: As models gain capabilities, they require more configuration options, making interfaces bloated and user experience worse.
• The stability assumption is broken: Traditional product design assumes stable underlying technology; AI models evolve too fast for conventional UX patterns to keep pace.
• Product builders must design for flux: Success requires abstractions that hide model complexity while remaining flexible enough to leverage new capabilities as they emerge.
• The answer isn't feature parity: Chasing every new model capability leads to unusable products; strategic capability selection and progressive disclosure are essential.

The Paradox Nobody Warned Us About

We're living through something unprecedented in software history. Every few months, foundation models get measurably better at reasoning, coding, vision, and dozens of other tasks. You'd think this would make building AI products easier. Instead, it's making it harder.

I've watched product teams celebrate a new model release, then spend the next six weeks wrestling with how to expose its capabilities without turning their clean interface into a settings nightmare. The problem isn't that the models are getting worse—it's that they're getting better in ways that break our assumptions about how software should work.

Simon Willison recently captured this tension perfectly in his piece "Better Models: Worse Tools". He describes how each generation of models brings new parameters to tune: temperature, top-p, frequency penalty, presence penalty, response format, tool use configurations, and more. What started as a simple text-in-text-out interface has metastasized into a control panel that would make a 747 cockpit look minimalist.

But here's what makes this particularly challenging for product builders: we can't just ignore these new knobs. They often unlock genuinely valuable capabilities that users will eventually expect. The question isn't whether to expose them—it's how to do so without destroying the user experience that made your product valuable in the first place.

Why Traditional Product Design Fails Here

Most product design operates on a fundamental assumption: the underlying technology is relatively stable. You might add features over time, but the core capabilities don't shift dramatically month-to-month. This stability lets you craft opinionated interfaces, make smart defaults, and guide users through well-understood workflows.

AI products operate under different physics. The foundation models they're built on are improving at a pace that makes traditional product cycles look glacial. GPT-4 to GPT-4 Turbo to GPT-4o to o1—each iteration doesn't just make existing things faster or cheaper. They change what's possible.

This creates several compounding problems:

The Configuration Explosion

Every new model capability potentially requires user configuration. Want to use the new vision features? You'll need to decide on image resolution, detail level, and how to handle multi-image inputs. Excited about structured outputs? Now you're defining JSON schemas. Interested in tool use? Welcome to function definitions, parameter descriptions, and execution policies.

The math is brutal: if you have 10 toggleable features and each has 3 settings, you're looking at 59,049 possible configurations. Most will produce garbage results, but which ones? Your users are now explorers in a vast configuration space with no map.

The Expertise Gap

Early adopters of AI tools often have technical backgrounds. They understand concepts like temperature and token limits because they've read the model documentation. But as these tools reach mainstream users, that knowledge gap becomes a chasm.

How do you explain to a marketing manager that they should use temperature 0.3 for factual content but 0.8 for creative brainstorming? More importantly, should you be explaining this at all? The moment you require users to understand model internals to use your product effectively, you've failed at product design.

The Moving Target Problem

You spend three months building the perfect interface for GPT-4's capabilities. You've made smart tradeoffs, hidden complexity behind intelligent defaults, and user-tested everything. Then GPT-5 drops with native multimodal understanding that makes half your carefully crafted abstractions obsolete.

This isn't hypothetical. I've seen teams rebuild core product flows three times in a year because new model capabilities fundamentally changed what users should be able to do. The traditional product development cycle—research, design, build, test, iterate—assumes you're working toward a stable end state. With AI, the end state is always six months away from not existing anymore.

My Take: We're Designing for the Wrong Layer

Here's where I think most product builders are getting it wrong: we're trying to design interfaces for models when we should be designing interfaces for outcomes.

The reason traditional software has stable interfaces isn't because the underlying code never changes—it's because the problems being solved remain constant. Excel's ribbon interface works because "sum these cells" is a stable user intent, even if the calculation engine underneath has been rewritten multiple times.

AI product builders need to make a similar leap. Stop exposing model parameters directly and start building outcome-oriented interfaces. Users don't want to set temperature to 0.7—they want to "make this more creative" or "stick to the facts." They don't want to configure JSON schemas—they want "give me this data in a format I can use in my spreadsheet."

This means building a translation layer between user intent and model configuration. Yes, this is harder than just exposing the raw parameters. Yes, you'll need to make opinionated choices that won't work for everyone. But that's what product design is—making informed decisions about tradeoffs so users don't have to.

The alternative is what we're seeing now: tools that are technically powerful but practically unusable for anyone who isn't willing to become an amateur prompt engineer.

Strategies for Building in the Flux

If you're building AI products today, you need strategies that account for continuous model evolution. Here's what's actually working:

1. Progressive Disclosure Over Feature Parity

Don't try to expose every model capability on day one. Build a simple, opinionated interface that handles 80% of use cases perfectly. Then add progressive disclosure for advanced features—but only when users demonstrate they need them.

Notion AI does this well. The basic interface is just "write" or "improve." Advanced options like tone and length exist but are tucked away. Most users never see them, but power users can access them when needed.

2. Intelligent Defaults That Learn

Your product should ship with defaults that work for most users, but those defaults should evolve based on actual usage patterns. If a user consistently tweaks temperature higher for certain types of prompts, your system should learn this and adjust.

This requires instrumentation and feedback loops, but it's the only way to handle the configuration space without drowning users in options.

3. Abstraction Layers That Survive Model Changes

Build your product architecture with a clear separation between user-facing abstractions and model-specific implementations. When a user asks for "creative" output, that should map to a configuration profile that you can update as models evolve.

This means when GPT-6 arrives with different optimal parameters, you update the mapping—not the user interface. Your users shouldn't need to relearn your product every time OpenAI ships an update.

4. Opinionated Workflows Over Blank Canvases

The most successful AI products aren't the ones with the most flexibility—they're the ones with the clearest workflows. GitHub Copilot doesn't ask you to configure code generation parameters; it just suggests completions in context. Midjourney doesn't expose all of Stable Diffusion's knobs; it provides a curated parameter space optimized for visual quality.

Opinionated design isn't about limiting power users—it's about providing a clear path for everyone else. You can always add escape hatches for advanced configuration, but the default path should require zero understanding of how models work.

5. Design for Capability Discovery

Users can't ask for features they don't know exist. As models gain new capabilities, your interface needs mechanisms for discovery that don't rely on reading release notes.

This might mean contextual suggestions ("I notice you're analyzing an image—want to compare it with another?"), progressive feature unlocking based on usage patterns, or even just well-timed tooltips that highlight new possibilities.

The Uncomfortable Truth About AI Product Design

Here's what keeps me up at night: we might be in a transitional period where truly great AI product design is temporarily impossible.

Think about it. The models are improving fast enough that any interface optimized for today's capabilities will be suboptimal in six months. But they're not improving fast enough to reach a stable plateau where we can design for a known end state. We're stuck in the worst of both worlds—constant change without clear direction.

This doesn't mean we should stop trying. It means we need to embrace a new design philosophy: build for adaptability rather than optimality. Your product won't be perfect for any single model generation, but it can be good enough for all of them if you design the right abstractions.

The teams that win in this environment won't be the ones with the most features or the ones that ship fastest. They'll be the ones who figure out how to provide consistent value while the ground shifts beneath them.

What This Means for Product Strategy

If you're leading an AI product team, this paradox has strategic implications beyond just interface design:

Resource allocation changes. You need to invest more in abstraction layers and less in feature-complete implementations. The code that maps user intent to model configuration is more valuable than the code that calls the model directly.

Competitive moats shift. In traditional software, features create moats. In AI products, the moat is increasingly about the quality of your abstraction layer—how well you hide complexity while maintaining power. This is why companies like Cursor and Replit are winning despite building on the same foundation models as everyone else.

User research becomes continuous. You can't do a research sprint, design an interface, and call it done. You need ongoing feedback loops to understand how users' mental models evolve as model capabilities change. What confused users six months ago might be intuitive now, and vice versa.

Documentation is a product feature. When your product's capabilities are constantly expanding, documentation isn't just nice-to-have—it's core to the user experience. Users need to discover what's newly possible without drowning in options.

Looking Forward: Will This Get Easier?

The optimistic view: model capabilities will eventually plateau, or at least slow their rate of improvement. When that happens, we can return to traditional product design cycles and build stable, refined interfaces.

The pessimistic view: we're entering a permanent state of flux where the tools we build are always slightly mismatched to the capabilities available. Product design becomes less about crafting perfect experiences and more about managing continuous adaptation.

My honest assessment? We're probably looking at another 2-3 years of high turbulence. Model improvements are still dramatic enough that they regularly invalidate product decisions. But we're also getting better at building abstraction layers that can absorb some of this change.

The products that survive this period will be the ones that embrace the paradox rather than fighting it. Better models do make tools worse—if you design them the same way you'd design traditional software. But if you design for continuous evolution, for outcome-oriented interfaces, and for progressive capability discovery, better models can make your tools genuinely better too.

The challenge isn't the models. It's our assumptions about what software products should be. And that's a much harder problem to solve than any technical limitation.

The Path Forward

For product builders working in AI today, the message is both daunting and liberating: the rules have changed, but they've changed for everyone. Nobody has figured this out yet. The companies that look like they're winning are mostly just iterating faster and breaking things less catastrophically.

This means there's still room for new approaches, new abstractions, and new ways of thinking about how humans should interact with AI capabilities. The teams that figure out how to build stable user experiences on top of unstable technology won't just build successful products—they'll define what AI product design means for the next decade.

The paradox of better models making worse tools isn't a bug in our current moment. It's a feature of the transition we're living through. The question is whether we're building products that can evolve through it or products that will need to be rebuilt every time the models take their next leap forward.

Choose wisely. The ground is still shifting.