Which agent bricked prod?

Every hop an agent makes has to answer two questions: whose authority is being exercised, and which agent exercised it. Most enterprise agents in production have already lost at least one of them, which means the auditability claim is fiction — and the role model, the permission documentation, and the incident-response plan are fiction along with it. That is the contrarian sentence; the rest of this post is what it looks like in wiring.

A while back I argued on this site that identity is the control plane: probabilistic recognizers are sensors, and the controls that actually hold are identity scoping, sandboxing, and a human on the irreversible actions. That post made the case in the abstract. This series spends it down in the place where it is genuinely hard — an estate with on-prem Active Directory, Entra ID, and a Keycloak nobody is allowed to turn off — and shows how the principle breaks the moment you stop drawing diagrams and start wiring tokens.

By “which agent” I mean the concrete thing that acted — the deployed workflow, the bot instance, the tool-calling runtime — not the app registration it happens to share with forty others. That distinction is the whole game, because the principal collapses in three different places and each erases a different half: the user disappears behind an app-only token or a service account; the agent disappears behind a platform-wide client identity; and the authorization disappears when data is copied out of the system that knew the ACLs into one that doesn’t. The rest of this post is those three collapses, in the order you meet them building the thing.

This part is the server-side case: the full-stack app with LLM calls behind it, the smolagents-shaped service, the low-code “agent” a colleague built last quarter. The agents that run on your device — Claude Code, the IDE assistant — are a different problem with a different trust boundary, and they get their own post.

The invariant, before anything else

A user must never silently gain authority by acting through an agent. The qualifier matters, because users trigger actions they can’t perform directly all the time — a deploy, an approval-gated purchase order, a provisioning workflow — and that is fine when the action is openly a service-owned operation, governed by its own policy and attributed as such. What is never fine is laundering: a user-initiated, user-attributed, user-audited action that is actually carried out with broader, non-user credentials behind the curtain. The moment an agent quietly does something the human driving it could not — while the books still read as if the human did it — every sentence you have written about roles and least privilege is void, because the agent is now the way around all of it.

State it operationally so it survives contact with real systems. When an action is represented as the user’s delegated authority, its effective permission after policy must be no greater than the intersection of what the user could do directly against that resource and what the app was delegated consent to request. When it is genuinely service-owned automation, don’t dress it up as the user. Either is defensible. The unauditable middle — the user’s name on the service’s power — is the whole disease. And mind that the intersection is not a property OAuth gives you: scopes are not ACLs and not business policy. A delegated token can carry a broad read scope and still return only the items the user’s own ACL allows — if the resource bothers to enforce on the user, which plenty of APIs don’t. The danger is the hop where the resource stops seeing the user at all, or never did.

Hold that next to the two questions. Preserving authority answers “on whose behalf.” It does not answer “which agent,” and you need both. Keep them in mind; every section below is the same invariant breaking in a different place.

The easy part is the boring part

Signing the user in is solved. Register the web app in Entra, put App Service Authentication (“EasyAuth”) in front of it, run the OIDC authorization-code flow, and you get a properly authenticated end user with almost no code. This is the part vendors demo. It is also the least interesting thing that has to go right.

One sharp edge worth naming, because it is where the rest of the post starts: EasyAuth terminates the sign-in for you and hands your app a set of injected headers. The ID token proves who signed in — it is not a key to anything downstream. To call another API as the user you need the access token, which App Service exposes as X-MS-TOKEN-AAD-ACCESS-TOKEN (with a token store and refresh path you have to actually configure and use). If you skip that and grab the ID token, or mint a fresh app-only token instead, you have already silently dropped the user. And because EasyAuth authenticates at the front door and hands your code the result in headers, the app has to be unreachable except through that front door — code that trusts injected identity headers on a directly reachable port is trivially spoofable. Everything that follows depends on preserving that user-bound access token and exchanging it — not blindly forwarding it — for each new audience.

Two doors to the second app

Now the app has to call a second Entra-protected API — say a line-of-business service holding the data the agent reasons over. There are two doors, and they are not interchangeable.

Application permissions (the client-credentials grant). The app authenticates as itself with a client secret or certificate, gets a token whose authority shows up as roles, and requests the .default scope after an admin has consented. There is no user in this token. The downstream API sees the application, not a person — identified by its client/app id, and only if it bothers to log and authorize on that. This is correct, and I want to be precise about it: for genuinely app-owned background work — a nightly indexer, a reconciliation job, a daemon that owns its own data, cross-tenant aggregation no single user is entitled to — application permissions are right. Pretending that work is a person would be the lie. The sin is not the grant. The sin is using it for a user-initiated action and then claiming user-level auditability you threw away at the token endpoint.

On-Behalf-Of (the delegated door). The app takes the user’s access token and exchanges it at the token endpoint — grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer, requested_token_use=on_behalf_of — for a new access token whose audience is the downstream API.¹ That token carries the user as oid + tid (use those; upn is mutable, sometimes absent, and a display aid, not an identifier) and the delegated scopes as scp. The downstream API sees the app acting as the user. Two preconditions people skip: the incoming token’s aud must be your API, and the downstream scopes must already be consented — no consent, no exchange. And OBO is delegated-only by construction: an app-only token has no user in it, so there is nothing to exchange. The two doors don’t open onto the same room.

Door one loses the principal entirely. Door two keeps the user half of it — and, as it turns out, the immediate caller, but not the chain behind it. Hold that “user half” qualifier, because it is about to matter.

The hop nobody plans for

The second app has its own agent now. It calls a third app. Where does the user go?

If the middle tier keeps using OBO, the user-bound chain can in principle continue — Entra does support exchanging again from a middle tier — but every additional tier needs its own app registration, its own exposed scopes, its own consent, its own audience handling, and it runs face-first into Conditional Access claims challenges and audience mismatches. A single tier with a custom token-signing key silently breaks the whole chain, because the next hop can no longer validate the signature. “Just keep exchanging” is a sentence; in an estate it is a quarter of work, which is exactly why people don’t, and reach for a client secret instead. The moment the middle tier calls the third app with application permissions, the user-bound chain terminates inside the middle tier. The third app sees a service principal with whatever that service was granted — frequently more than the user ever held. That is the invariant breaking, quietly, one tier in.

Here is the part worth being honest about, because it is where I see the most over-claiming. The clean answer — carry the actor — is real as a standard and mostly absent as a deployment. RFC 8693 token exchange describes the cleanest shape for it: a token with the user as subject and the agent as actor (act, with may_act able to express who may act for whom — enforcement still depends on the authorization and resource servers honoring it), so a downstream service can see both “this was Alice’s authority” and “exercised by agent X” in one trace. That is the tidiest standard form, not the only correct design; real estates often carry the actor in a compensating trace outside the token instead. Entra OBO is not that. It is Microsoft’s own JWT-bearer flow, it predates and sits beside RFC 8693, and it does not hand you act/may_act actor chains in ordinary downstream tokens. So in a normal Entra estate you can preserve the user across a hop or two with real effort, and the resource can see the immediate caller — the middle-tier app — in azp/appid. What it cannot reconstruct is the chain behind that caller: Alice, via agent X, in platform Y, through middle-tier B. You get the user and the last app; you lose the actor chain, and the actor chain is the half the incident bridge asks for first.

There is a second failure mode that has nothing to do with token types, and it is the one I find most under-appreciated: trust drift. Consent is granted to scopes, not to a frozen behavior. You integrated with a well-behaved internal API last year, consented to exactly what it needed, signed off. This quarter its team shipped “agent” features and it now fans your users’ delegated access out to a dozen systems it never touched before — under the consent you already gave, because the scopes did not change, only the code paths behind them did. Nothing renegotiated. Nobody re-consented. You authorized a connection; what you are exposed to is that connection’s future blast radius, and there is no Entra event that fires when it grows. Permission governance helps less than you’d hope here: app-consent reviews and permission reports can tell you a scope changed or that an app looks risky, but none of them fire when the behavior behind an unchanged scope quietly grows.

Down to the metal: when the target is on-prem AD

Eventually some service has to read or write against a directory-integrated system on the old network. The lazy version is everywhere: one service account, a simple LDAP bind with a bind DN and a password, broad rights, and every action the agent ever takes attributed to that single account forever. The principal doesn’t just degrade here — it collapses to one identity that is neither the user nor the agent.

The honest version is Kerberos, and it is worth being exact about what bridges what, because “the user logged in with OIDC at the web tier” does not grant anyone Kerberos powers. The KDC does not accept an Entra token and mint tickets from it. What happens is that a service account you have trusted for constrained delegation with protocol transition — classic or resource-based, the S4U shape is the same — asks the KDC, via S4U2Self, for a ticket to itself on behalf of a named AD user, no user password involved, and then via S4U2Proxy for a delegated ticket to a specific backend SPN it has been explicitly allowed to reach. The bind to LDAP is then SASL/GSSAPI with that ticket, not a simple bind. The backend can finally authorize as the user — the user half of the trace, preserved all the way to the directory. (The agent half is still missing: Kerberos carries the user and the delegating service, not “agent X acted through service Y.”)

The thing nobody puts on the slide is the join in the middle: the Entra user has to be an AD principal for any of this to resolve, and that means more than identifier equality — UPN alignment via Entra Connect, or an explicit, trustworthy mapping from the cloud-authenticated subject to the on-prem impersonation target. That hybrid identity plumbing is the hard, brittle part. And note what you have built to get here: a service account trusted to impersonate arbitrary users to named services. That is a loaded capability. Done with tightly scoped SPNs and monitoring it is far safer than a god-mode bind account; the user-side controls exist for exactly this risk — marking sensitive accounts as “cannot be delegated,” putting them in Protected Users — and the classic way it goes wrong is the lazy upgrade to unconstrained delegation, or an SPN allow-list nobody has pruned since 2019. The reason teams skip all of it and use the fat account is not ignorance. It is that the correct thing is genuinely painful, and the painful thing is the only thing that preserves the user.

The legacy corner: Keycloak

Some apps don’t speak Entra; they speak to the Keycloak that was supposed to be decommissioned two reorgs ago. The real shape here is brokering: Keycloak trusts Entra as an upstream identity provider over OIDC or SAML — metadata and signing-certificate exchange, claim and role mapping — and then issues its own tokens to the legacy apps behind it. (If your estate genuinely still runs WS-Federation in this corner, that is the oldest path and it works, but it is not where new trust should be built; OIDC/SAML brokering is.)

Every problem above still applies, and Keycloak adds a fresh way to break the invariant at the mapping layer. A sloppy mapper takes a narrow Entra group claim and lands it on a broad Keycloak realm role; or it gives every authenticated user a default legacy-user role that happens to imply write. Now Keycloak hands out a token broader than anything the user holds upstream — the second IdP has laundered privilege, cleanly, with a green check in the brokering config. Brokering a login is not the same as preserving authority across it, and token exchange inside Keycloak (configuration- and version-dependent, and easy to set to “impersonate” when you meant “delegate”) will happily widen the audience while you are not looking.

Where this actually lives: agent platforms

Almost nobody writes the bespoke app above. They use a platform — Copilot Studio, Power Automate, n8n, dify — where a developer stands up the platform and non-developer “makers” build the actual “agents” on top. The auth models differ a lot between these, and the platform tends to normalize the differences badly, so resist the urge to treat them as one thing; the pattern is shared even where the products aren’t.

The current enterprise-agent wave mostly entered through “chat with your documents” — an “agent” was a namespace in Azure AI Search plus some uploaded files and a system prompt — then grew connectors into SharePoint, Confluence, and Jira, and now, because couldn’t it also do something, takes actions and talks MCP. MCP is not an auth model, by the way; it is a way to expose tools and context. It doesn’t break your authorization. It surfaces the credential boundary you already failed to design.

And you failed to design it a year ago, on the read side, before anyone added a single write. The common low-code and DIY RAG build ingests SharePoint, Confluence, and Jira through document intelligence under one broad crawler credential, chunks and embeds the text into a shared index, and retrieves from that index with no per-user security trimming. The source ACL is a property of the system you copied out of; it is not in the chunk — and there is no convenient place to put it back. Upload files as knowledge to a Copilot Studio agent and they are flat: everyone who can reach the agent reads everything you ingested, full stop. Reach for Azure AI Search instead and you can carry a per-document ACL field and trim on it at query time — but that copy is frozen at index time and never syncs back to the origin, so the day someone loses access at the source your index keeps handing it to them. The only version that actually holds is to keep the origin URI on every chunk and re-check the caller’s live access against the source system on each retrieval: correct, and brutal — an extra authorization round-trip per hit, against the same SaaS APIs that already rate-limit you, times every query from every user. At a hundred thousand seats nobody enables that by default. So the index ships flat, or it ships stale, and flat is what’s in production. Share that bot with a colleague and they can now read, through it, things they could never open at source. The authorization failure is complete before the agent can do anything but talk. And the store you built is invisible to the tooling your privacy office runs on: Purview eDiscovery and Priva subject-rights search reach Exchange, SharePoint, OneDrive, and Teams — they do not reach an Azure AI Search index.² One chunk of PII in there is personal data your DSAR and legal-hold workflows cannot enumerate, redact, or produce — a shadow copy of the corpus with the ACLs stripped and the compliance plumbing unplugged, and GDPR’s upper tier prices that exposure at up to 4% of global turnover.

Then the maker adds write — create tickets, update lists, publish pages — and the same broken model gets hands. How that write is wired runs down a depressingly short ladder. At the bottom, and far too common, the maker pastes in a high-privilege API key, sometimes a personal access token, and every action the agent takes belongs to that one credential. Jira and Confluence don’t consume Entra OBO at all — and the deeper problem there isn’t a missing flow, it’s that the identity domains don’t compose: Entra’s notion of Alice and Atlassian’s notion of Alice are different objects with different authority, and nothing reconciles them but a human pasting a token. One rung up, the platform actually preserves the human at the target — often by holding a per-user OAuth refresh token for the SaaS behind a single shared connector app — but the agent collapses into that one platform identity: the client id is identical for every agent every maker ever built, and the resource is left correlating on platform-internal run IDs that, more often than not, were never propagated to its logs. At the top, rare enough that I would not assume it exists unless you can point to both halves in the downstream logs, each agent has its own identity and clean per-user delegation — and even then the resource only sees the full trace if someone propagated a correlation id or modeled the actor outside the token, because the token alone will not carry it.

Down to where the value is actually made

Read everything above as the warm-up. For an AI-first enterprise, the prize is not another assistant that rewrites slides. The prize is the shop floor. That is where the company’s margin is made — a line that produces, a batch that yields, a setpoint that decides whether the next twelve hours meet spec or scrap. If your chatbot can summarise an OKR document but it cannot ask is line 3 going to make the morning deadline and adjust the schedule when the answer is no, your AI program has not reached the part of the business that pays for it. The fully-integrated version of the strategy deck connects agents to MES, to the historian, to the SCADA HMI, to the recipe executor. There is no realistic version of that deck where it does not, and the roadmaps already reflect it. What that closes is the same loop the rest of this post has been about — whose authority, which agent — except that the chain now terminates somewhere that physically moves.

The layer those connectors land on is not the layer the rest of this post prepares you for. The system on the other end has no concept of Alice asked for this; it has a concept of the right thing is on the wire, and it logs accordingly. Whose authority and which agent are not first-class concepts in Modbus or in the historian’s bulk-write API — the protocols and conventions at this tier were not designed for the kind of IT-style per-action delegation an agent assumes. Modbus has no authentication at all. OPC Classic rides DCOM, and the Windows identity it ends up using has, in many estates, collapsed to a shared operator or service login the integrator pasted in years ago. Plenty of PLCs and HMIs still gate on a single panel password — sometimes the one printed in the manual. OPC UA, by contrast, has a sane identity story with per-user certificates and session establishment, and the moment your estate runs serious UA and the integrator wired it per-user rather than under one shared client cert, most of this stops applying. The operative claim is not “all OT is laminated passwords.” Mature estates have validated MES workflows, electronic batch records, domain-integrated HMIs, and safety layers that take change seriously. The claim is narrower and harder to dismiss: the tier the agent roadmap intends to reach was built around different controls — the air gap, then the Purdue-model boundary, then operational mediation — and “IT/OT convergence” has been the polite word for dissolving those for fifteen years. The dissolving is the point. The dissolving is what makes the AI-first manufacturing story possible at all.

Hand an enterprise agent a connector into that tier and the two questions stop being unanswered and start being unrepresentable. “Whose authority” collapses to network position. “Which agent” collapses to whatever account the integrator put in the script. Every action ever taken through that path attributes, in the historian, to one login a dozen people share and three more know the password to. That is not a logging gap. That is the field by design — and the field is exactly what the AI roadmap intends to reach.

A plant-floor reader will object, fairly, that this never makes it to a real setpoint. There are DMZs and read-only historian replicas, there is management of change, there are interlocks and safety-instrumented systems, and there is a culture that does not trust a model to twist a valve. All true, all real controls. The trajectory is also real, and it is how every integration of this kind has gone in the last two years. The first version is read-only — the agent reads tags and answers questions about yield. The second is and could it also open a maintenance ticket. The third is and could it adjust the next batch’s parameters, within the validated window. Each step is defensible in isolation; each step accretes a write path through scheduling, recipes, maintenance, or batch parameters; and none of those steps re-asks whether the chain that issues the write can name the person and the agent behind it. A misrouted agent write to a Jira ticket is a noisy email. A misrouted agent write to a setpoint, or a recipe row in a batch executor, moves actuators. The blast radius of which agent, on whose behalf, did the thing stops being the books and becomes the line, and on a bad day, the people next to it. OT teams have lived with shared-credential pain for decades and built compensating practice around it; what is new — and what the AI-first roadmap specifically introduces — is putting an LLM-driven actor on the IT side of a chain that already could not tell you who acted, and pointing it at the part of the company that physically moves.

Which agent bricked prod?

Here is what actually happens, stripped of charity. The technical user the agent runs as is over-privileged, or it is an API key, and someone shrugs and lets it rip. It works, until the day it doesn’t.

Alice asks the internal agent to “tidy up stale tickets before the release.” It does — with the maker’s admin PAT, across projects Alice can’t even see — and one of the tickets it bulk-closes is the change-freeze hold on a build whose forward-only schema migration had been flagged “not yet reviewed.” Closing the hold removes the gate. The deploy pipeline takes the green light, cuts the build, runs the migration against prod, takes an exclusive lock on the orders table, and finishes in a state the rollback playbook cannot undo because the migration is one-way. The site is down for three and a half hours while the on-call DBA restores the orders table from the previous night’s backup and reconciles the gap from the write-ahead log by hand. Now reconstruct the decision. Jira’s author field says the maker. Entra’s sign-in logs have nothing tied to the change, because it never went through Entra as Alice. The platform has a run ID nobody forwarded anywhere the resource could see. So you reconcile sign-in timestamps across three systems whose clocks don’t quite agree, trying to prove which agent, on whose behalf, lifted the hold — and from the authoritative logs you can’t. You can guess from chat transcripts and adjacent timestamps; you cannot prove it. The forensics aren’t hard. They’re impossible, by construction.

That is the part that should worry a regulated shop more than any jailbreak. Financial-record provenance regimes — Germany’s GoBD, the internal-control attestations behind SOX, the audit expectations baked into SOC 2 and ISO 27001 — all want the same property: a firm has to be able to show how an entry in its books came to be. Provenance, not vibes. When a shared key made the change and you can’t tie it to a person acting within their rights, you don’t just lose the one entry; you lose confidence in the ledger around it. The property that fails the auditor on a quiet Tuesday is the same one that fails the incident bridge during an outage: if you cannot separate agent from user, your logs are not evidence, they are material for an argument you will lose.

Why this hasn’t blown up yet

Every failure named in this post has, so far, been bounded by accident. The maker’s PAT only flattens what the maker happened to wire it into. The flat RAG index leaks what one team crawled, not the company. The shared OT login lights up one plant, not the next. The agents do not talk to each other yet, and the compartments are doing all the work.

That is not a control. It is the property of the technology being eighteen months old. The compartment walls are about to come down, because the business case for agent-to-agent writes itself in a sentence: the onboarding agent calls the IT-provisioning agent calls the facilities agent calls the payroll agent, no human in the loop, in seconds. The first time a board sees that demo, the second time someone insists their competitor already runs it, the question is settled. It will ship.

The day it does, every one of these flatten-and-collapse patterns turns transitive. An attacker with a foothold in any one agent does not need a payload, a binary, a C2 channel, or a CVE — they need polite REST calls to the next agent over. Living off the land becomes living off the chatbot: lateral movement is the published API, and privilege escalation is whichever neighbour has a broader connector. The onboarding agent now holds, transitively, the union of every connector every agent it can reach ever held — and because none of those connectors carry a faithful actor chain past their first hop, nothing downstream of the breach can tell that the action it just executed was the seventh hop of an attack rather than the first.

The narrow claim, the one I am willing to defend, is this: commercial pressure will push inter-agent orchestration into production before actor-chain controls are mature. That is enough; the rest follows. The case against — “we cannot audit it” — sounds, in the room, like the people who said the same thing about cloud, mobile, and SaaS, and the room mostly stopped listening to them. So A2A ships, and the first incident makes the policy retroactively. The discovery phase will feel familiar to anyone who has worked one of the big transitive-dependency post-mortems of the last decade: wait, that thing could call this thing?, across weeks, while the answer to which agent, on whose behalf is a shrug and a timestamp adjacency. The bar I am about to state for Part 1 is going to read, in two years, as the floor we should have built before we let them call each other.

So the bar for Part 1 is lower than any architecture diagram, and it just restates the two questions as an evidence requirement. If your system cannot show, from its own authoritative logs, whose authority an action carried and which agent exercised it — along with the credential identity it ran under, what it was authorized to do, and what it actually did downstream — then you do not have agent authentication. You have a shared account with an LLM bolted onto it. The next posts are about closing that gap on purpose. This one is just about admitting how wide it is.