Explainability Doesn’t Scale in Agentic Systems. It collapses.

Sponsored by

Looking for unbiased, fact-based news? Join 1440 today.

Join over 4 million Americans who start their day with 1440 – your daily digest for unbiased, fact-centric news. From politics to sports, we cover it all by analyzing over 100 sources. Our concise, 5-minute read lands in your inbox each morning at no cost. Experience news without the noise; let 1440 help you make up your own mind. Sign up now and invite your friends and family to be part of the informed.

Subscribe to 1440 today.

Hello everyone and welcome to my newsletter where I discuss real-world skills needed for the top data jobs. 👏

This week I’m discussing a problem with multi-agentic systems. 👀 

Not a subscriber? Join the informed. Over 200K people read my content monthly.

The actual number is 250K last month and since February of 2107 I’ve amassed 73.6 millions views across multiple platforms. I’m not trying to flex here but that puts me in the top 1% of all authors in the world of artificial intelligence. Join the enlightened. 🤓

Thank you. 🎉

Move 37 

In 2016, AlphaGo — the AI system developed by DeepMind — played a historic match against Go world champion Lee Sedol. And in the second game, on move 37, AlphaGo did something no human had ever done. The model made a move so strange, so statistically improbable, that commentators assumed it was a glitch.

It wasn’t. It was genius.

Move 37 broke the game wide open. It was the move that won. But when engineers and experts tried to understand why AlphaGo had made it, the answer was simple: they didn’t know. It hadn’t reasoned its way there. It hadn’t strategised. It had simply run the numbers. Move 37 had the highest win probability, so AlphaGo played it.

It wasn’t risk-taking or intuition, but rather a pure statistical emergence. A product of playing itself millions of times and favoring the numbers that worked. The move was brilliant, but the system that made it couldn’t tell you why — and for most machine learning models that’s brilliant, that’s not a bug, but a feature.

The Problem With Agentic Systems 

Welcome to 2025. AI is now embedded in nearly every corner of every organization, moving faster than the mechanisms meant to monitor it. Guardrails are scarce. Transparency is rarer still. Once again, we’re the experiment. The live test environment.

And what’s coming makes the last wave of AI adoption look tame.

AI is no longer just responding to commands. It’s beginning to act — on its own, without oversight, and increasingly in collaboration with other AI agents. We're not just inputting data and getting summaries anymore. We're watching autonomous systems delegate tasks, make decisions, negotiate outcomes, and execute actions — all without human intervention.

This is the era of agentic AI.

Agentic models or combinations of models don’t wait for instructions. It pursues objectives. It reasons (statistically), plans (probabilistically), and acts (autonomously). That includes everything from sending calendar invites and processing invoices to drafting reports and selecting which tools to connect with next — all in service of completing the task at hand.

Google’s latest announcement introduces its A2A (Agent2Agent) protocol — a framework that allows AI systems to communicate and coordinate directly with one another, executing tasks across platforms, APIs, and services without any human in the loop.

And Google isn’t the only one pushing this forward. Workday describes it as the AI-to-AI handshake — a future where intelligent agents trigger and manage enterprise workflows like an invisible network of ultra-efficient middle managers.

With so many agents primed for deployment, you’d think we’d have robust tools to audit what they’re doing. But we don’t. 

We already struggle to explain how a single AI model reaches a conclusion. Now picture chains of decisions made by swarms of agents, each trained on different data, optimizing for different objectives, and operating across separate systems — all at a speed that outpaces human comprehension. Observing it is hard. Interpreting it is harder still.

As Dr. Adnan Masood, Microsoft Regional Director and Chief AI Architect at UST puts it, AI-to-AI interactions operate at a speed and complexity level that makes traditional debugging, logging, and inspection approaches almost useless.

Start learning AI in 2025

Keeping up with AI is hard – we get it!

That’s why over 1M professionals read Superhuman AI to stay ahead.

• Get daily AI news, tools, and tutorials

• Learn new AI skills you can use at work in 3 mins a day

• Become 10X more productive

Sign up and start mastering AI

Explainability Doesn’t Scale in Agentic Systems — it breaks.

So instead, we get a façade. A performance. A wave of tools now claims to deliver Explainable AI, offering users and stakeholders polished, plausible stories. But that’s all they are: stories. Rationalizations, reverse-engineered to sound convincing — not to reveal the actual logic beneath. As the IT consultancy Veritis puts it, These tools don’t offer rationales. They offer results.

And that distinction matters. Because when AI agents begin to coordinate — executing tasks, controlling access, making decisions on behalf of institutions — we’re not just dealing with opaque outputs anymore. We’re dealing with systems that resist accountability.

In human decision-making, we can trace responsibility: Who approved this? Who made the call? Who’s accountable? With agentic AI, the answer is often no one. Or worse: everyone, everywhere, all at once.

We don’t know what these agents are optimizing for. We don’t know how they communicate. And most dangerously, we don’t know what they’re willing to sacrifice in pursuit of efficiency.

It’s insane how fast we moved — from predictive text to distributed decision-making. From automating tasks to outsourcing judgment. From asking How did this happen? to wondering What made this happen in the first place?

AI-driven decisions already shape much of daily life — often invisibly, yet profoundly. They determine who gets a job interview, who gets flagged for security screening, what price you see for a flight, and whether your loan is approved. And soon — much sooner than most expect — they’ll influence whether a self-driving car swerves left or right, sacrificing one to protect many.

These systems aren’t just supporting our decisions anymore — they’re replacing them. Bit by bit, we’re offloading human judgment. We’re trading cognitive effort for convenience. And in doing so, we’re giving up more than just control — we’re surrendering autonomy.

Thanks for reading and have a great day. 👏