AI won’t replace product teams, but it will expose how broken their information systems are

For the past two years, the loudest question hanging over product organizations has been blunt and existential: Will AI replace us? Product managers. Designers. Engineers. Whole teams, flattened by a wave of automation and “agentic” tools that promise to think, plan, and build on our behalf.

According to Chris Butler, Director of Product Operations at GitHub, that framing misses the real story. AI isn’t threatening product teams. It’s holding up a mirror to them and exposing how fragile their information systems already are.

“We’re taking systems that already involve multiple humans,” Chris explains, “and we’re adding LLMs into them.” The problem isn’t that AI can’t think like a human. It’s that most product organizations don’t have a shared, reliable way of capturing how humans already think, decide, and collaborate.

What breaks first isn’t headcount. It’s context.

Product teams aren’t broken. Their information flows are.

Modern product organizations run on a dense web of artifacts. Issues, pull requests, specs, prototypes, Team meetings, Slack threads, Word Docs, SharePoint, Google Docs, roadmaps, dashboards. Each one captures a fragment of intent or decision-making, often created by different people at different moments for different audiences.

GitHub itself is a good example. “It’s a multi-person interaction,” Chris says. “PRs, issues, discussions – it’s meant to be between people.” Over time, those interactions form an informal operating system for the team. Humans learn how to navigate it by experience. AI does not.

When teams introduce AI into this environment, they often expect magic. What they get instead is friction in the form of missing context, contradictions, outputs that feel plausible but wrong and, increasingly, security and privacy constraints that limit what context AI systems are even allowed to see. As Butler notes, a growing amount of time is spent securing approval for the integrations that would make AI genuinely useful in the first place.

That’s not a failure of the AI. Rather, it’s an information design failure.

Chris describes product teams as systems of information flows – systems that were never designed to be clean, explicit, or machine-readable. AI doesn’t create the mess, it simply reveals it.

Why “agentic AI” is mostly a distraction

Few terms have spread faster, or meant less, than ‘agentic’, Chris argues. It’s become shorthand for AI that acts independently, makes decisions, and gets work done end-to-end.

But Chris is blunt about the reality. “We keep saying agentic, but we don’t really mean agentic,” he says. Most so-called agents aren’t acting anonymously roaming freely in product organizations. They’re being triggered by human actions at specific moments within a workflow.

True agency, he argues, looks very different. “I can go off and do work by myself for a day or two and then come back and update an issue thread. That to me would be more agentic.”

What teams usually want isn’t autonomy, but assistance. They want LLMs that draft documents, summarize meetings, turn transcripts into issues, or help refine a spec. These tools don’t replace people, but instead sit inside human systems and depend on them to function.

The danger of the agentic narrative is that it encourages teams to skip the harder work of clarifying ownership, decision points, and context. Without those, no amount of autonomy helps.

Why reporting still fails

One of the most persistent failures in product organizations is reporting. The status updates, leadership summaries, and progress decks that are written, rewritten, and reinterpreted at every level of the org.

Chris argues the problem here isn’t effort, but rather direction. “The way organizations work today is that information gets pushed,” he says. Teams create updates and hope they reach the right people at the right time.

But that model breaks down fast. People forget, context gets lost, and leaders often don’t visibly engage with the updates they receive. Over time, that creates the sense that information is being sent into a black hole, which quietly degrades the reporting process itself, as teams stop believing the effort is necessary or valued.

AI opens the door to a different approach. Not more automated reporting, but less of it. Instead of pushing information upward, leaders should be able to pull what they need from a shared information ecosystem, shaped by their role and decisions.

“That’s actually not as helpful as the opposite,” Butler notes. “Based on what I need as a leader, I want to pull this information out of the ecosystem and engage with the teams that need help or direction before they have to do an official escalation.”

Again, this isn’t an AI problem. It’s a product operations problem centered around structure, visibility, and trust.

Faster prototypes haven’t replaced judgment

AI has made it dramatically easier for product teams to turn ideas into something tangible. Functional prototypes, “vibe‑coded” interfaces, and working flows can now be generated in hours rather than weeks, often before teams have fully agreed on what problem they’re solving.

Some have taken this as a sign that traditional artifacts – such as specs, PRDs, and documentation – are obsolete. But Chris disagrees.

“I think prototypes are very helpful in having discussions,” he says. But they aren’t answers. They’re provocations. “Here’s how I’m thinking about it. Now let’s critique it and start to unearth assumptions that haven’t been written down.”

Prototypes compress time. They don’t eliminate complexity. Engineering still has to worry about maintainability, security, accuracy, and long-term evolution. Product still has to make bets about the future and set the intent for the product.

Continuing on this theme, the AI accelerates artefact creation, but it doesn’t remove the need for human judgment.

From vibes to evals

As AI tools become more embedded in product workflows, they change not just how work gets done, but who is doing what. Many teams are spending less time creating first drafts and more time reviewing, correcting, and approving AI‑generated output.

But there’s a limit to how far that scales.

“If generation is taken away from people for rudimentary things, that’s probably okay,” Chris says. “But now they have become a critic. And can you do critique for 40 or 60 hours a week? Probably not.”

This is where many teams get stuck. They rely on intuition – or vibes – to decide whether outputs are good enough. That might work early on, but it breaks down pretty fast.

Over the past year, Chris has seen a growing focus on evaluations, such as prompt evals, output quality checks, harm evaluation, and red teaming. He sees this not as bureaucracy, but as survival.

“Vibes don’t help you build and ship maintainable products,” he says.

Product Ops plays a critical role here. Not by writing every evaluation, but by defining what good looks like. What should be measured? What risks matter? What signals are meaningful as models change?

AI is decomposing roles, and Product Ops is the glue

Despite the fear, Chris doesn’t expect product team structures to change dramatically in the near term. Job titles will stay and disciplines will remain. What will change is the work inside them.

AI breaks roles into tasks. Some automate cleanly, but others don’t. The danger is that humans are left doing only the hardest parts, such as constant review, exception handling, or judgment calls under uncertainty.

Without thoughtful design, it leads to burnout, not efficiency.

Product Operations then becomes the connective tissue deciding which tasks are automated, where humans stay in the loop, and how information moves without overwhelming the people inside the system.

AI doesn’t fix chaos. It amplifies it.

It's a given that AI will keep improving, and models will get faster, cheaper, and more capable. But none of that solves the underlying problem most product teams face.

If context lives in people’s heads and decisions are scattered across tools, if information only moves when someone remembers to push it, then AI will surface those weaknesses faster, not hide them.

The teams that succeed won’t be the ones chasing the most autonomous tools. They’ll be the ones who treat product work as an information system first.

Because AI doesn’t replace product teams. It reveals them.

eBook

Beyond the buzzwords:
How to win in 2026

Read now