MCP server OAuth

When an MCP server needs OAuth, Zimmer runs the whole flow itself (discovery, registration, PKCE, token exchange, refresh) and then writes the tokens into the agent CLI’s own credential file so the agent’s MCP client finds them.

That last step is why this is harder than it sounds: Zimmer has to produce a file in a format that another vendor’s private code will read.

The gate

Before spawning, McpOauthCredentialInjector#check_credentials_status looks at every remote MCP server on the session (http / streamable-http / sse):

A session that needs OAuth fails fast: it goes to failed with failure_reason: oauth_required instead of hanging or prompting, and the UI turns that into Authorize buttons. Completing the flow resumes it — with the turn it was blocked on, which is not always its first (which prompt the resume delivers).

The post-spawn MCP-failure classifier (AgentSessionJob#check_and_handle_mcp_failure) applies the same rule. An auth-shaped error (401, Unauthorized, Supported scopes, invalid_token, …) only becomes oauth_required when the server is actually OAuth-capable — McpOauthCredentialInjector.oauth_capable_server?: in the catalog, remote transport, and no static credential header.

A “static credential header” is decided by the header’s name, matched against a word list (CREDENTIAL_HEADER_PATTERN): authorization, auth, api-key/apikey, token, secret, password, credential(s), as whole -/_-delimited parts. Vendors spell that header however they like — X-API-Key, X-Goog-Api-Key, X-Figma-Token, PRIVATE-TOKEN — and the spelling says nothing about whether an OAuth flow exists, so matching words beats matching names. The list stays narrow on purpose: key counts only within api-key, so Idempotency-Key is not a credential, and auth must be a whole part, so X-Author is not one either. Reading a routine header as a credential would hide the Authorize button on a server that genuinely needs one, which is the worse failure of the two.

A static-header server (e.g. Zimmer’s own zimmer* entries, which send X-API-Key: ${ZIMMER_PROD_API_KEY}) returns the same 401 when its token is invalid or under-scoped, but no OAuth flow can mint a valid API token, so it is never routed to a dead-end Authorize button. It is left out and the session keeps running, with the raw error and the credential to check written into the session log — see When a server cannot connect. This is the single predicate shared with the pre-spawn gate above.

That split is the whole fatality policy: oauth_required is the one failure class that still stops a session, because it is the one a human can resolve by clicking Authorize. Every other class is definitive — no amount of waiting or authorizing changes it — so stopping buys nothing and costs the transcript.

There is a second dead-end the classifier avoids: a server Zimmer already holds a valid credential for that still returns 401. That is not a missing authorization — it is the runtime failing to honor the token Zimmer injected, most often because Claude Code’s host-global negative-auth cache (~/.claude/mcp-needs-auth-cache.json) short-circuited the connection (Skipping connection (cached needs-auth)) before it ever reached the network. Routing it to oauth_required is pointless: McpOauthController#initiate short-circuits on the existing credential, so the Authorize button can only redirect straight back — which reads to the user as “the button does nothing”. So the classifier (and the OAuth banner, and the initiate controller) all consult McpOauthServerAuthorization.authorized?, and a failure for an already-authorized server instead clears the runtime needs-auth cache and retries, so the next spawn reconnects with the token already on hand. That retry survives a different server in the same handshake failing definitively: the classifier degrades and retries per server, so a co-failing static-credential rejection no longer writes off the server whose cache was just cleared before the retry the clear exists to enable ever happens (#689). Injecting a credential (McpOauthCredentialInjector#inject_credentials!) always clears that cache entry for the same reason, and the OAuth banner filters oauth_required_servers through the same predicate so a stale entry (e.g. a recovery job cleared failure_reason but left the list behind) never renders an Authorize button that cannot resolve.

That “we already hold a credential” check asks whether a row exists and is unexpired — which is not the same question as “the provider still honors it”. So one class of failure is carved out ahead of it: when the error says the provider rejected Zimmer’s refresh grant — Claude Code reports Token refresh failed with invalid_grant: Invalid refresh token — the stored credential is permanently dead no matter how unexpired the row looks. Retrying can never revive it; without the carve-out the server was filed as “already authorized” and rode the retry ladder into a terminal mcp_connection_failed, orphaning the session with no Authorize button (#222). Instead the credential is retired and the server is routed to oauth_required — which now resolves, because the short-circuit in initiate no longer sees an active credential. Only cache- and transport-shaped auth failures keep the clear-cache-and-retry path.

Retiring takes two stores, not one. McpOauthServerAuthorization.invalidate! drops the revoked refresh token and force-expires the DB row — force-expiring the access token too, deliberately: a runtime refreshes ahead of expiry, so the paired access token may have minutes of TTL left, and those minutes buy nothing once the credential is terminal while leaving the row active is exactly what re-shadows the Authorize button. But the runtime’s copy still carries its original future expiry, so McpOauthRuntimeReconciler would read it as a strictly newer pair and adopt the dead tokens back into the DB on the next spawn. So the classifier also calls delete_credentials on the runtime credential writer, leaving nothing to adopt.

REFRESH_TOKEN_REJECTED_PATTERN keys on the refresh-failure phrasing (Token refresh failed with <grant error>, or Invalid refresh token) rather than on a bare invalid_grant anywhere in the text. A server that brokers a downstream OAuth of its own can report its provider’s invalid_grant while Zimmer’s credential for that server is healthy, and retiring it there would force a re-auth that cannot fix anything. An unrecognized phrasing costs nothing — it falls through to the retry path.

Which prompt the resume delivers

oauth_required is not exclusively a first-turn failure, so a resume is not exclusively “replay the stored prompt”. Three doors reach failed + oauth_required, and two of them can do it to a session that is already owed a message a human sent:

Door	What it fails	What the session is owed
AgentSessionJob’s pre-spawn gate on a first turn	a session whose prompt was never delivered	nothing — the stored prompt is the turn
AgentSessionJob’s pre-spawn gate on a follow-up (“Follow-up blocked: OAuth authorization required for MCP servers”)	the turn carrying a human’s message	that message
Edit MCP servers / Edit plugins (Sessions::UpdateCatalogSelection)	an idle session, never a running one	whatever undelivered turn it was already holding

A follow-up exists only as its AgentSessionJob’s argument. The job returns when the gate blocks, nothing re-queues it, and the resume builds a job of its own — so before failing the session the gate hands the prompt back as pending_follow_up_prompt, the marker every recovery path already reads as “handed to a job, not yet delivered”, and says in the session’s timeline that the message has not been delivered yet. A nudge is the exception and is deliberately dropped: AutomatedPrompts.nudge? is the class of prompt that says nothing the session does not already know — the heartbeat beat, “you may have been interrupted, carry on” — and HeartbeatSweepJob refuses to stamp its own beat for the same reason, that a beat delivered for a moment which has already passed is worse than one not delivered. McpOauthResumeService#resume! then asks that marker first:

• owed a turn → Session#deliver_follow_up! — the session resumes to running and the job carries that prompt;
• owed nothing → AgentSessionJob.enqueue_new_session — the session goes back to waiting and the stored prompt is replayed, exactly as it always did.

Before #887 the resume always took the second branch, so a human who sent a follow-up, saw the Authorize banner, and authorized got their session back running its original prompt, with the message dropped and nothing in the log saying so — and on a session with a clone that meant re-running work it had already done.

The one case the resume cannot honour is a session with no runtime session_id: there is no conversation for a follow-up to continue, and AgentSessionJob would reclassify it as a fresh start and run the stored prompt anyway. It resumes the first turn and writes a warning into the session’s own timeline quoting the text so it can be re-sent. Saying so is the point — a silent substitution is the defect this closes. It also gives up custody of the marker in the same breath, moving the text to oauth_undelivered_follow_up_prompt, a key the service owns and nothing consumes: a standing pending_follow_up_prompt promises a delivery, and CleanupOrphanedSessionsJob reads that promise as a delivery in flight and stops reaping the session while it waits for one that is not coming.

What the server advertised, recorded

Two places actually ask a remote server whether it requires OAuth: the spawn gate above, and McpOauthProbe, which re-checks a session’s servers whenever its MCP-server or plugin selection changes. Both run RFC 9728 / RFC 8414 discovery and an unauthenticated GET against the server URL. The Connectors page cannot afford a network call, and used to fall back to a guess: a remote server with no static credential header and no stored token might need OAuth, so assume it does.

McpServerOauthRequirement is where the answer is kept, so the surfaces that cannot ask read a recorded fact instead. One row per server config — keyed on the same {type, url, headers} digest as the credential, so a catalog edit that invalidates the credential key invalidates the determination with it — carrying one of three values:

Determination	What it means	Recorded when
advertised_required	The server advertises OAuth	Discovery resolved an authorization server, or the unauthenticated GET came back 401 with a Bearer challenge
advertised_not_required	The server served us without a token	The unauthenticated GET returned 2xx and answered in application/json or text/event-stream
undetermined	We asked and could not tell	A connection error, a timeout, a 401 whose challenge is not Bearer, or any other status

undetermined still means what the guess used to mean: assume OAuth might be required. That fallback is deliberate and load-bearing in the other direction. Deciding “this server does not need OAuth” and being wrong is the worse failure, because it is silent — the server never gets credentials and fails at the point of use, where an over-eager assumption merely offers an Authorize button nobody needed. Two rules keep the quiet direction narrow:

• Only an MCP-shaped 2xx records advertised_not_required. A 400, 404, 405 or 5xx is undetermined — a streamable-HTTP server that answers a bare GET with 400 missing session id before it ever looks at a token has told us nothing about authorization. Neither does a 200 in text/html: the probe sends a bare GET with Accept: application/json, which is not a well-formed MCP request, so a 2xx is at least as likely to be a CDN interstitial or a landing page sharing the origin as it is to be the MCP endpoint. The content type is what makes the 2xx evidence rather than coincidence.
• advertised_not_required expires after seven days and degrades back to undetermined, so a server that starts requiring OAuth cannot be permanently believed not to. advertised_required does not expire; the failure it causes is a visible button, not a silent one.

Nothing about which servers block a spawn changes: the gate still probes live and acts on its own answer, and the Connectors page still offers the Authorize button on every OAuth-capable server with no credential. What the record buys is that the state is inspectable. “The agent says it needs to authorize this server” has three distinct causes — this guess, a credential key that stopped matching, and a server that issues no refresh token — and a Needs authorization row now says which branch it took: silent when the server advertised the requirement, and explicit that Zimmer is assuming when nothing has come back. The spawn log says the same thing, naming “advertised: not required” apart from “could not determine whether OAuth is required” rather than reporting both as “does not require OAuth”.

The authorization flow

Public clients and manual (paste-back) completion

Two capabilities let Zimmer authorize against servers that expose a public OAuth client (no client secret) and only permit a localhost / out-of-band redirect — the motivating case being the official hosted Slack MCP server (https://mcp.slack.com/mcp), which is designed to be used with Slack’s own app client_id + a localhost redirect + PKCE, and which deliberately does not support DCR.

Public clients (no client_secret)

A mcp_oauth_clients entry — or a catalog oauth block naming only a clientId — may omit the client secret entirely. Such a client is a public client (RFC 6749 §2.1) that proves possession with PKCE alone (RFC 7636). The token exchange omits the client_secret parameter when the flow has no secret and relies on the persisted code_verifier; when a secret is configured, the previous client_secret_post behavior is preserved.

Every outbound call McpOauthService makes — probe, discovery, DCR, and the token exchange itself — sets open_timeout and read_timeout to REQUEST_TIMEOUT (30 seconds). McpOauthCredential#refresh! posts its refresh grant through the same helper, so the unattended path carries the same bound as the interactive one. An auth server that accepts the connection and then never answers fails the exchange rather than holding a request thread — or, on the cron path, a GoodJob thread.

The token endpoint must be https

token_endpoint decides two things at once: where the grant goes, and whether it is encrypted — McpOauthService#post_form derives use_ssl from that same string. Both grants posted through it carry the client_secret as a form parameter, alongside the authorization code (initial exchange) or the refresh token (renewal). So an http:// value publishes long-lived credentials in a cleartext POST body, and the endpoint answers 200, so the refresh reports success and nothing surfaces.

Zimmer refuses it, and the rule is exceptionless — loopback included. That matters more here than for the X credential, because this endpoint is discovered: it comes out of the MCP server’s own RFC 8414 metadata. An exception would be triggerable by the one party outside Zimmer’s control, and a remote server naming Zimmer’s own loopback has no honest meaning — it would aim a POST carrying an operator-supplied client secret at whatever answers on this host’s port. A local MCP server on http://localhost is refused too, with a message saying why, rather than quietly putting the secret on the wire.

Where it is enforced:

• McpOauthController#initiate — refuses before the consent redirect, with a flash naming the endpoint, so no authorization code is minted for an endpoint Zimmer will not talk to.
• McpOauthPendingFlow — refuses to store one, so the initial exchange cannot leak it.
• McpOauthCredential — refuses to save one. Blank stays legal: it means “this credential cannot be renewed, re-authorize it”, which is what #can_refresh? and the Connectors page read.
• McpOauthCredential#can_refresh? — false for a cleartext endpoint, so RefreshMcpOauthTokensJob and McpOauthCredentialInjector never call refresh! on one. This is the anti-brick guarantee: refresh! POSTs before it saves, so a row that reached it would leak the secret, let the provider rotate the single-use refresh token, and only then fail validation on the way to persisting it.
• McpOauthService#post_form — raises InsecureEndpoint rather than opening a cleartext connection, covering any caller holding a bare URI.
• McpOauthService#post_json — the same rule on the registration endpoint. Dynamic Client Registration is the same hole one request earlier: the endpoint comes from the same discovery document, and the DCR response carries a freshly minted client_id and client_secret. It runs inside check_oauth_requirement, i.e. before initiate’s check, so the guard has to be in the transport. A refusal degrades to the existing “Dynamic Client Registration failed” message.

ClearNonHttpsMcpOauthTokenEndpoints clears any row that predates the rule. It clears to NULL rather than to a default, because an MCP token endpoint has no default — re-authorizing the connector rediscovers it.

Userinfo is deliberately permitted (https://id:secret@host/token): post_form reads it as basic auth on purpose, because some providers document client_secret_basic that way, and over TLS it is as confidential as any other header. That is where this rule differs from XOauthCredential’s, whose transport overwrites the Authorization header and would silently ignore it.

Manual (paste-back) completion

Some public clients only permit a redirect URI they already whitelisted — for the official Slack client that is http://localhost:3118/callback, the loopback redirect the Claude Code Slack plugin uses. Zimmer’s hosted callback (https://<host>/mcp_oauth/callback) cannot be added to someone else’s app, so those flows complete out-of-band:

1. redirect_uri comes from the statically-configured redirect — the catalog oauth.redirectUri or a mcp_oauth_clients entry, whichever configures the server — rather than the hosted callback. Because Zimmer cannot receive that redirect, the flow is marked manual.
2. initiate renders a paste-back page instead of redirecting: it shows the authorize link and an input for the redirect URL.
3. You open the authorize link, consent in your own browser, and land on the localhost redirect with nothing listening — that failed page load is expected; the value you need is in the address bar (?code=…&state=…).
4. You paste that full URL (or the bare code) back. complete extracts the code, validates state against the persisted flow (the same CSRF check the hosted callback does), and finishes the exchange using the persisted PKCE code_verifier and redirect_uri.

Configuring the official Slack MCP

The official Slack app is a public client. Wire it up entirely through credentials — no code change per server:

mcp_oauth_clients:
  slack:
    client_id: "1601185624273.8899143856786"        # official Slack app (public; not a secret)
    authorization_endpoint: "https://slack.com/oauth/v2_user/authorize"
    token_endpoint: "https://slack.com/api/oauth.v2.user.access"
    scopes: "channels:history,groups:history,search:read.public,users:read"
    redirect_uri: "http://localhost:3118/callback"    # the loopback redirect the Slack app permits
    manual: true
    resource: ""                                      # Slack OAuth is not RFC 8707 — suppress the indicator

The key (slack) must match the MCP server name in the catalog, whose URL is https://mcp.slack.com/mcp. Slack returns the user token nested under authed_user.access_token (a top-level access_token, when present, is the bot token); Zimmer unwraps it on both the initial exchange and the cron token refresh, so a rotation-enabled credential survives past its first expiry. resource: "" is set because Slack’s OAuth endpoints are not the RFC 8707 audience-binding kind — for a genuine MCP auth server, omit the key instead and the pre-registered path derives the resource indicator from the server URL automatically.

The Slack client_id is public; wiring it into prod is a human step

1601185624273.8899143856786 and the 3118 loopback redirect are taken from the distributed Claude Code Slack plugin — they are public identifiers, not secrets. Writing the entry into config/credentials/production.yml.enc still requires the prod master key, so it is a one-time human step. The code path and its tests do not depend on live Slack.

The credential key is a copy of Claude Code’s private algorithm

To make the agent’s MCP client find the token, Zimmer must key it exactly the way Claude Code keys it. McpOauthCredential.compute_credential_key:

"#{server_name}|#{SHA256(compact_json({type, url, headers}))[0,16]}"

…where “compact JSON” is faked by string-munging ": " → ":" and ", " → ",", and streamable-http is normalized to http. A canary test pins the literal key for two fixed configs — notion|3fad03f7abd02b9c for {"type":"http","url":"https://mcp.notion.com/v1/mcp","headers":{}} — so that a change on Zimmer’s side of the algorithm fails a test instead of silently missing every credential lookup.

This is a reimplementation of another project’s internals

It is a hash algorithm reverse-engineered from Claude Code so the two agree on a dictionary key, with no documented format and no API behind it. If Claude Code changes how it computes that key, every MCP OAuth credential Zimmer holds becomes unfindable — and the failure mode is silent: the agent just says it needs authorization.

Codex is worse. CodexMcpCredentialWriter’s format was read out of codex-rs/rmcp-client/src/oauth.rs @ rust-v0.133.0, and it writes two different, mutually incompatible schemas — the file uses server_url + a scopes array + millisecond epochs, while the macOS Keychain path uses url + a nested token_response + a space-delimited scope. The Keychain path has never been runtime-verified (“Zimmer’s CI/staging/production workers are all Linux”).

And it only exists because of two open Codex bugs

CodexMcpCredentialWriter’s header explains why Zimmer rewrites Codex’s entire MCP credential store on every session spawn:

• openai/codex#15122 — credentials from codex mcp login don’t persist across restarts.
• openai/codex#17265 — Codex won’t use the stored refresh token, so MCP calls fail with “Authorization required.”

So Zimmer refreshes the tokens itself every 30 minutes and re-writes them at spawn, so Codex never has to. It’s a workaround for someone else’s bugs, and it will need to be removed when they’re fixed.

Pi takes a token by handoff, and cannot give one back

PiMcpCredentialWriter is the third writer, and it exists because an OAuth-credentialed MCP server was otherwise unusable on Pi. pi-mcp-adapter answers a connect to one with an instruction to run mcp({ action: "auth-start" }) or /mcp-auth <server> — and neither route exists in an unattended session, where there is no browser and nobody to click.

The adapter has a documented ingest path, so Zimmer writes into it: a plaintext entry at $MCP_OAUTH_DIR/sha256-<server-hash>/tokens.json, or under <Pi agent dir>/mcp-oauth/ when that variable is unset, where <server-hash> is SHA-256 of the .mcp.json server key. On the next read the adapter imports it into the OS credential store and deletes the file, which is the intended lifecycle rather than a leftover. Every field comes from ResolvedMcpCredential; nothing is synthesized.

The handoff goes both ways. Zimmer gives Pi a token it already holds, and reads one Pi refreshed for itself back out of the credential store — Pi ships its own MCP OAuth client and rotates like Claude Code does, so the write half alone would leave a rotating provider’s credential stale. See capturing the token the runtime rotates, and the limitation that does remain — nothing pushes a refreshed token into a session that is already running.

Refresh

RefreshMcpOauthTokensJob, every 30 minutes. It refreshes credentials expiring within an hour — but throttled by PROACTIVE_REFRESH_MIN_INTERVAL (won’t touch anything updated in the last 4 hours), deliberately, to reduce exposure to rotating-refresh-token reuse detection.

It splits network errors carefully:

• Retryable — the connection was never established, so the server never saw the request. Safe to retry in-band.
• Ambiguous — the request went out and the response was lost. Never retried in-band; deferred to the next cron run. Retrying could burn a single-use refresh token.

That distinction is the kind of care that’s easy to skip and expensive to skip.

A refresh is treated as permanent when the token endpoint rejects the refresh_token grant with any 4xx — the refresh token is dead and re-auth is required. Most servers signal this with a spec-compliant JSON body ({"error": "invalid_grant" | "invalid_client" | "unauthorized_client"}), but some return a bare HTML 400 Bad Request, so the 4xx status code — not the body — is what classifies it. On a permanent failure it nulls the refresh token but keeps a still-valid access token instead of force-expiring it. Transient failures — 429 rate-limits and 5xx outages — are excluded first: the refresh token itself is not implicated, so it is left intact and the failure stays on the loud ERROR log path (which pages #alerts) to retry on the next cron run. This transient / permanent split matches XOauthCredential.

Capturing the token the runtime rotates (write-back)

Zimmer is not the only party that refreshes these tokens, and it cannot become the only one. Claude Code and Pi both ship their own MCP OAuth client: when an access token lapses mid-session they refresh it and write the new pair back — Claude Code to ~/.claude/.credentials.json, Pi (pi-mcp-adapter) through an MCP-SDK OAuth provider whose saveTokens writes into the OS credential store. Notion (and other OAuth 2.1 servers) rotate refresh tokens — every refresh mints a new refresh token and revokes the prior one — so once either runtime refreshes, the refresh token in Zimmer’s DB is already dead.

Neither runtime can be told to stop. pi-mcp-adapter’s only relevant switch is oauth: false, which disables OAuth for the server outright rather than pinning the token Zimmer staged, and no environment variable narrows it; Claude Code exposes nothing either. So “Zimmer is the sole authority, runtime refresh disabled” is not a configuration that exists — adopt-back is the design, not a workaround for one. The ordering it needs is supplied by the adoption rule below rather than by a lock: only a strictly later access-token expiry is adopted, so the chain advances in one direction and a re-stamp of an older pair can never win.

ClaudeMcpCredentialWriter#merge_preserving_fresher! protects that fresher on-disk entry only while its paired access token is still valid. Across an idle gap longer than the access token’s TTL (~1h for Notion) the on-disk access token lapses, so on the next spawn Zimmer’s stale DB entry wins and clobbers the good on-disk refresh token. The next refresh — Claude Code’s at connect time, or RefreshMcpOauthTokensJob’s from cron — then presents the revoked token and gets invalid_grant: Invalid refresh token, and the server drops offline until a human re-authorizes.

McpOauthRuntimeReconciler closes that loop. Before Zimmer refreshes or injects a credential it reads the runtime’s on-disk store (RuntimeMcpCredentialWriter#read_runtime_credentials) and, if the runtime holds a strictly newer token pair — a later access-token expiry means the runtime refreshed after Zimmer last wrote the row — adopts that pair into the DB. Crucially it adopts even when the on-disk access token has already expired: a rotated refresh token is the live head of the chain regardless of its paired access token’s TTL, which is the exact case merge_preserving_fresher! drops. ClaudeAccount#sync_tokens_from_filesystem! does the same thing for the runtime’s own account tokens; MCP OAuth credentials had no equivalent, which is why they went stale.

The reconciler runs in two places:

• McpOauthCredentialInjector, on every spawn, before it decides whether to refresh or gate the session — so a session never injects (or re-auth-prompts against) a rotated-away token. It reads the store of that session’s runtime.
• RefreshMcpOauthTokensJob, before the cron refreshes each credential — so the cron adopts a session’s rotation instead of burning the stale DB token against the provider’s reuse detection. The cron has no session and therefore no runtime, so it reads every registered runtime’s store (RuntimeRegistry.mcp_credential_writer_classes). Reading only one is how a rotation performed on another runtime got burned. Order does not matter: each store is compared against the row as it stands after the previous one, so the newest pair wins whichever order they are read in.

Which key each store uses is the runtime’s own, and all three differ. Claude Code keys by the protocol-level credential_key (server_name|hash), because its #credential_key_for is McpOauthCredential.compute_credential_key. Codex uses the same shape over a different hash — it forces type: "http" and empty headers where the protocol key hashes the server’s real type and headers, so the two coincide only for a headerless streamable-http server and diverge for an sse one or one carrying a header. Pi keys by the bare .mcp.json server name. A caller holding a server config asks #credential_key_for; the cron, which holds only a DB row, asks #runtime_key_for, and each writer answers in its own shape. The shared contract test asserts the two agree per writer, because a writer whose key shapes disagree probes its store for something it never wrote and misses in silence.

How the store gets read differs too, and the contract says which kind a writer is. #enumerable_store? is true for a runtime that keeps one readable file (Claude Code, Codex): the reconciler reads it once and serves every credential from that snapshot. It is false for Pi, whose entries live in the OS credential store addressed by sha256(server_name) with no listing — so #read_runtime_credentials takes the keys the caller wants and probes exactly those accounts through the adapter’s own mcp-keyring-helper.cjs, memoized so a repeated key costs one probe. A store that cannot be reached is “nothing to adopt”, never an error: an adoption probe is bounded at two seconds (tighter than the write and delete paths, because it is the only one that can be skipped without consequence) and a failure leaves Zimmer’s own copy in place.

Two things are refused rather than adopted, both of which would otherwise overwrite a live token with a dead one. A DB row with no expiry is not adopted over: a nil expires_at means the provider issued no expires_in, the access token does not expire, and a non-expiring token is one no runtime ever refreshes — so reading “the runtime recorded an expiry and we did not” as newer would let a stale on-disk pair land on top of a freshly authorized one. And an entry recorded against a different server URL is not this credential’s, however its key matched: server_name is not unique across McpOauthCredential rows, so on Pi’s name-keyed store a server whose URL changed leaves an old row probing the same account. pi-mcp-adapter draws the same line from its own side (getAuthForUrl returns nothing once the URL has moved).

Which store it reads depends on the session-scoped credentials setting. With it off, one host-global ~/.claude/.credentials.json that every session on the worker read-modify-writes under a flock. With it on, ClaudeMcpCredentialWriter.for_session points the writer at that session’s own CLAUDE_CONFIG_DIR — same keys, same adoption rule, one writer per file, so the read-modify-write stops racing. The cron and the revocation path still target the host-global file: they have no session to scope to, so a revoked credential is not removed from a session that is already running (it gets a fresh directory next time).

Codex is the one runtime with nothing to adopt: it is written-not-trusted (Zimmer rewrites its store every spawn and Codex does not refresh MCP tokens itself), so reconciling against it is a harmless no-op rather than an exception to carve out.

What adoption is not is a push. Credentials are injected at spawn, and nothing writes a refreshed token into a session that is already running — nor would it take effect if it did. pi-mcp-adapter memoizes each server’s auth entry in a process-local cache and drops it only when its own provider rejects an access token, so a mid-session write is invisible to the live process until it next hits a 401 — at which point it re-reads and picks up the newer token anyway. Adoption is therefore periodic: every spawn, and every 30 minutes from cron. That is early enough that nothing goes stale, because the cron never presents a rotated-away token; it is not a push. See limitations.

This is also what makes an OAuth MCP connection survive a worker/clone recreation. When a session is recovered after a deploy or restart, the relaunch goes through the follow-up spawn path, which re-injects credentials and re-writes .mcp.json before spawning claude --resume. Before the write-back existed, that relaunch re-injected the stale DB refresh token, so Claude Code’s reconnect refreshed against a rotated-away token, got invalid_grant, and the server came back with all its tools reporting No such tool available. The restart didn’t break the token — it forced the reconnect that exposed an already-stale one. With the reconciler, the relaunch injects the token the previous run rotated to, and the server reconnects.

Seeing where every connector stands

/connectors — Connectors in the left-hand nav — lists every MCP server in the catalog with its current auth status, one lazily-loaded Turbo Frame per server so the list renders before any status resolves. It replaced the older “OAuth Status” page, which could only show servers that already had a credential row and so was silent about precisely the servers that needed attention.

ConnectorStatusProbe reads the same three inputs a spawn reads, in the same order, so a connector reported Ready is one that will actually connect:

State	Meaning
Ready	OAuth is complete and the credential saved, or every required ${VAR} resolves
Needs authorization	An OAuth-capable server with no stored credential. The row carries an Authorize button, and says whether the requirement was advertised or assumed
Token expired	Expired, but has a refresh token — RefreshMcpOauthTokensJob will renew it
Needs re-auth	Expired with no refresh token; the row carries a Re-authorize button that replaces the credential in place
Missing configuration	A required ${VAR} has no value. The row says where it goes — see The Secrets Console
Unavailable	The catalog entry carries an unavailable declaration — breakage no local check can infer. Nothing on the page fixes it; the entry has to change
Secret store unreachable	The store did not answer. Deliberately not “missing” — see Secrets in the Parameter Store
No credential required	The catalog entry configures no credential at all
Probe failed	Anything unexpected, isolated to that one row

One line cuts across the states rather than being one of them: a credential whose server issued no refresh token carries an amber note on its row saying so, in every state including Ready. Nothing can renew that credential, so the re-authorization is permanent and periodic — see Servers without offline_access below.

A credential filed under a different credential key than the catalog currently computes is deliberately not matched — the injector would not find it either, so counting it would report Ready for a server that cannot connect. Those show up instead under “Unclaimed credentials” at the bottom of the page, where they can be deleted.

The page never contacts the MCP server itself and never displays a secret value; it reports presence and where to set what is absent.

The same probe decides what anyone is offered. Missing configuration, Needs authorization, Needs re-auth and Unavailable block a spawn, so get_configs leaves those servers out of its MCP-server list and names them in a trailing Unavailable roster instead — one line and a compact reason each. Token expired does not block, because the refresh job resolves it.

The human surfaces read the same four states through McpServerOptions: the MCP-server pickers on the new-session form, the trigger form and the session detail page show such a server with an Unavailable badge and its reason, sorted below the ones that work, and GET /api/v1/configs and GET /api/v1/mcp_servers carry unavailable and unavailable_reason per server. They flag rather than omit because a human, unlike an agent, is usually one click on this page away from fixing the reason. → Availability, and what an agent is offered

How the list fills in, and why it re-orders itself

The rows ship as loading="lazy" frames and connector_list_controller then promotes them to eager — six at a time, releasing a slot as each frame loads (turbo:frame-load), comes back without a matching frame (turbo:frame-missing, which is what a 404 or an error page produces), fails at the network level, or hits a 15-second watchdog. All four matter: a row whose probe 404s never fires turbo:frame-load, so without the second listener it would hold its slot for the full watchdog and the sort would wait on it.

Each half of that is load-bearing:

• Lazy in the markup is the floor. Before the controller connects — and if it throws, or its bundle fails to load — the frames still resolve on appearance, which is what they did before. It is a floor, not a no-JavaScript fallback: Turbo’s lazy loading is itself JavaScript, so with none the rows never resolve either way.
• Promoting them is the fix. Turbo’s lazy defers a frame until it scrolls into view, so on a ~100-server catalog every badge below the fold stayed blank until you went looking for it.
• Six at a time is what keeps the fix from being a regression. Un-gating all ~100 frames at once fires ~100 requests at a Puma pool of a handful of threads, and the queueing makes the first badge slower than it was before.

Ordering follows from the same design. A server’s state is computed inside its own frame, so ConnectorsController does not know at index-render time which servers have problems; sorting server-side would mean probing all ~100 up front and holding the whole page on the slowest one — exactly what the frames exist to avoid. So the sort happens in the browser, once, after the frames settle: rows are alphabetical while they load, then re-order by severity with the problems first and a N need attention, listed first count in the header. Sorting during the load was rejected deliberately — it moves content under the reader for the whole load.

Severity itself is not decided in JavaScript. ConnectorsHelper::SEVERITY_RANKS maps each probe state to a rank, the resolved row carries it as data-connector-rank, the attention threshold is passed in as a value, and the controller only compares numbers. A test asserts the rank table covers ConnectorStatusProbe::STATES exactly, so a new state cannot quietly default into the healthy group.

Two edges are handled where they would otherwise be invisible. Turbo Drive’s page cache restores this list as the controller left it — resolved bodies, and the eager the controller itself wrote — so on a back-navigation onto a half-loaded page the controller pre-settles anything already carrying content and counts only frames it started; otherwise the in-flight count goes negative, the window blows open, and the list never sorts. And a reorder moves DOM nodes, which drops keyboard focus, so the sort stands down while focus is inside the list and retries once it leaves.

Authorizing from the Connectors page

A row that needs a consent screen runs one: Authorize (or Re-authorize on a needs_reauth row) POSTs to the same /mcp_oauth/initiate the session banner uses, just without a session_id. Authorizing a connector is something you do to Zimmer, not to one session, and it used to cost you a throwaway session to do it.

A session-less flow differs from an in-session one in exactly two places:

• It returns to /connectors rather than to a session — through the callback, through paste-back, and through every initiate error exit. (A callback that fails outright renders the shared OAuth error page, as an in-session one does.)
• It resumes nothing, because nothing is parked on it. McpOauthResumeService is skipped rather than called with no session.

Everything in between — discovery, DCR, PKCE, the hosted callback, the paste-back page, the stored McpOauthCredential — is the same code on the same path. The credential is keyed on the server config, not on who started the flow, so a connector authorized here is a connector every future session inherits.

Only rows where a consent screen is actually the fix get the button: needs_authorization and needs_reauth. A token_expired row does not — the refresh job resolves it without you. A missing_configuration row does not either: its credential is a ${VAR} secret and no OAuth provider will ever set it. Nor does a server authenticated by a static header, whatever the vendor named it: with its ${VAR} set it is ready, and with it unset it is missing_configuration — never needs_authorization, because there is no OAuth flow behind that button to run.

The button offers only catalog servers, and the session-less initiate enforces that server-side: the server must be in the catalog and pass McpOauthCredentialInjector.oauth_capable_server?. Both paths now read the server URL from the catalog whenever the catalog has the server, so a server_url submitted alongside a session-less initiate is ignored outright.

A session-less flow has no session to be reaped with (Session has_many :mcp_oauth_pending_flows, dependent: :destroy is what collects the in-session ones), so initiate calls McpOauthPendingFlow.sweep_expired_session_less! each time it starts a flow. An abandoned Connectors-page flow would otherwise sit indefinitely holding a PKCE code_verifier and a client secret.

X (Twitter) is minted from /supervisor

The X MCP server does not go through anything above. It runs in static access-token mode, and the token comes from an XOauthCredential row that XOauthTokenVendor resolves as ${X_OAUTH_ACCESS_TOKEN} at prepare time and RefreshXOauthTokensJob keeps fresh. X rotates refresh tokens single-use and expires them, so that row is not minted once and forgotten: a reused token, a double exchange or an expiry breaks the chain, the refresh fails permanently, and the only way back is a human consenting on X again.

That consent runs from the Supervisor panel (Supervisor::XOauthAuthorizationsController):

• Re-authorize with X on a credential’s page (/supervisor/x_oauth_credentials/:id) re-runs consent for that credential’s account and env var.
• Connect an X account on the index asks for an account key and env var, then does the same for an account with no row yet.

Things that hold on every path:

• Every leg is behind the operator realm, the callback included. The callback arrives in the operator’s own browser, which already holds the /supervisor credential, and the fleet’s sessions do not hold it (CliSpawnEnv clears SUPERVISOR_PASSWORD). So an agent can neither start a flow nor finish one. That is a tighter boundary than the MCP flow’s, which has none beyond the perimeter.
• The state X echoes back is the only way to a flow. A missing, unknown, replaced or expired state stops the request before anything is sent to X. Claiming deletes the row, so a replayed callback, a double-submitted paste and a second tab all find nothing. An expired flow is deleted when it is refused, and every start sweeps expired flows.
• The paste-back requires the whole redirect URL. A bare code is refused, because the state is what ties the code to the verifier that can redeem it.
• A failed exchange stores nothing. The code is single-use at X, so the page says to start again, and the credential Zimmer already holds is untouched.

Which of the two completions a deployment gets is decided by X_OAUTH_REDIRECT_URI: the flow finishes on its own only when that is byte-for-byte XOauthBootstrap.hosted_redirect_uri (https://<APP_HOST>/supervisor/x_oauth/callback). The default is http://localhost:8080/callback, the one URI the X app already has registered, which is why paste-back is what ships. To get the hosted callback, register https://<APP_HOST>/supervisor/x_oauth/callback on the X app in X’s developer portal, then set X_OAUTH_REDIRECT_URI to it. Registering it is a step on X’s side that no API reaches.

Consents in progress are listed at /supervisor/x_oauth_pending_flows (verifier not shown), where one can be cancelled. There is deliberately no MCP tool for any of this: consent is a browser handshake, and keeping the flow behind the operator realm is what keeps agents off it.

Known problems

Anyone who can reach the host can start an OAuth flow

McpOauthController has skip_forgery_protection only: [:callback, :initiate, :complete] — and Zimmer has no user authentication at all.

The state parameter is the only CSRF defense on the callback. On initiate, the defense is that the request cannot freely invent its target: whenever the catalog has the server, its URL is read from there rather than from the request, and a session-less initiate additionally refuses a server the catalog does not have at all. The gap that remains is an in-session initiate naming a server outside the catalog — that one still falls back to the submitted server_url, and discovery will fetch it.

The loopback check has no production caller

McpOauthPendingFlow#localhost_flow? parses redirect_uri and compares the host exactly against localhost, 127.0.0.1 and ::1; a malformed or schemeless URI is not a loopback. Nothing in the app calls it — the decision that actually matters, whether a flow can be completed automatically, is made by comparing the redirect against build_redirect_uri, below. The predicate is correct so that the first caller inherits a correct answer, not because it gates anything today.

Servers without offline_access issue one-shot credentials — and Zimmer says so

Scope acquisition just joins whatever the server advertises in scopes_supported. If a server doesn’t advertise offline_access, no refresh token is issued and the credential is single-use: nothing can renew it, so it becomes requires_reauth? the moment it lapses.

That is a permanent property of the server, and it is knowable the instant the token response arrives. A token exchange that leaves no refresh token on the credential sets refresh_token_unsupported, and McpOauthCredential#requires_periodic_reauth? (the flag, plus a still-absent refresh token — so one that arrives later settles the question) drives an amber line on the Connectors row saying the credential cannot be renewed and will need authorizing again. It is said while the row is still green, which is the only time saying it helps (#64).

“Leaves no refresh token” is the test, not “this response carried none”: plenty of servers mint a refresh token on first consent and omit it when re-authorizing a grant that is still live. A re-authorization that omits one keeps the stored token rather than nulling it, and the flag is derived from what survives — so a token exchange can never leave the row claiming renewable on a credential it just emptied, or one-shot on a server already seen to issue a refresh token (#309).

What has not changed: Zimmer does not ask for offline_access a server did not advertise, and the re-authorization is still manual. The fix is that a permanent limitation is stated once, up front, instead of resurfacing as “the agent randomly needs me to authorize this server again”.

client_id resolution order

McpOauthService resolves client_id in this order: (1) a statically-configured client id from the server’s catalog oauth block (oauth.clientId, camelCase; client_secret optional) — used verbatim and, when present, DCR is skipped entirely; (2) Dynamic Client Registration (RFC 7591) when the auth server advertises a registration_endpoint; (3) the literal "zimmer" fallback, used only when neither a configured client nor a DCR endpoint is available. The configured path exists for servers that require a pre-registered client and expose no usable DCR endpoint (e.g. Slack, whose slack-reframe catalog entry ships its clientId) — there, the "zimmer" literal is rejected outright (invalid_client_id). This is distinct from the fully-static PreregisteredOauthConfig (Rails credentials mcp_oauth_clients), which also supplies the authorization/token endpoints and bypasses discovery; the oauth.clientId path supplies only the client id and still discovers endpoints via RFC 8414/9728.

The catalog oauth block also configures the redirect URI (oauth.redirectUri, camelCase; redirect_uri accepted). It is read on the same footing as the client id, by the same resolution the PreregisteredOauthConfig path uses: when set it wins over Zimmer’s hosted /mcp_oauth/callback, and when absent the hosted callback stays the default — which is what every ordinary server uses. A pre-registered client only accepts the redirects registered against it at the provider, and for a public client Zimmer does not own that is a fixed URL it cannot change: Slack’s slack-reframe entry declares http://localhost:3118/callback, and handing Slack the hosted callback instead fails at the consent screen with redirect_uri did not match any configured URIs.

Paste-back is derived from the redirect URI, not a per-server flag

Zimmer can only finish a flow automatically when the provider redirects back to the callback Zimmer itself serves. So McpOauthService#manual_completion_required? marks a flow manual whenever its redirect URI is not build_redirect_uri — a third-party localhost URL or an oob URN lands somewhere Zimmer never sees, and the user completes it by pasting the resulting URL into POST /mcp_oauth/complete (manual (paste-back) completion).

The test is “is this our callback”, not “does this look like localhost”: with APP_HOST unset the hosted callback is itself http://localhost:3000/mcp_oauth/callback, which must stay automatic. A mcp_oauth_clients credentials entry can still force paste-back with manual: true regardless of its redirect URI; catalog-configured servers need no flag, because the declared redirect already says whether Zimmer can receive the callback.

Re-authorizing a server reaches a live session as a notice, not as a reconnect

Claude Code reads its MCP servers once, at launch. A running agent process therefore cannot be handed a connection it did not start with, and Zimmer does not pretend otherwise: the only mechanism that would is killing the process and starting another one, which is the double-process hazard #400 documents.

So McpOauthResumeService re-spawns only a session that was blocked on OAuth (failed with oauth_required, or waiting with oauth_required_servers). For a session that is running or needs_input and wires the server whose grant was just renewed, it does three other things (#195) and returns :reconnect_pending:

• injects the fresh credential into the runtime store immediately, rather than leaving it to be discovered at the next spawn, and clears the runtime’s needs-auth memo for it;
• records the server under metadata["mcp_oauth_reconnect"], which renders the session page’s “Authorization complete — reconnects on the next turn” notice with a button that sends (or, on a running session, queues) an ordinary follow-up message;
• writes a line into the session’s own timeline saying the grant is back and that the next message is what reconnects it.

The reconnect itself is the next turn’s gate_and_inject_oauth!, which is also where the notice is cleared — that spawn has injected the credentials, so the statement about the previous process is spent. The button is deliberately a follow-up on the ordinary path rather than a new resume path: enqueue_new_session replays the session’s original prompt, which is the wrong thing to say in the middle of a conversation.

The notice reaches the session the flow was started from, and only that one. A grant is keyed by credential key, not by session, so re-authorizing from the Connectors page renews it for every session that wires the server — and notifies none of them, because a session-less flow has no session to run the service against (see the Connectors caution below, which says the same thing about a parked session).

The replayed turn carries its attachments — when there is still a first turn to replay

When McpOauthResumeService resumes by re-queuing the original run, AgentSessionJob receives images and files exclusively as job arguments, so a bare re-queue replayed “here is the screenshot, fix this” with the prompt and without the screenshot. The resume now reads them back off the durable volume through Sessions::FirstTurnAttachments (#789) — but only when the session’s transcript is empty, meaning no turn has reached an agent yet. oauth_required is not exclusively a first-turn failure: Edit MCP servers and Edit plugins set it when a human adds an OAuth server to a live session, AgentSessionJob’s follow-up branch sets it under “Follow-up blocked”, and the post-spawn MCP-failure classifier sets it after the process has run. On a session that has already run, everything on its volume includes attachments earlier turns consumed, so the resume carries none — and the spawn it re-queues resumes the existing conversation rather than replaying the prompt, so there is no turn there to put them on.

A resume that delivers a held follow-up rather than the first turn carries none either, for the same reason read the other way: those attachments belong to the first turn, and this is not it. When there were any to leave behind, the session’s timeline says so.

Authorizing from the Connectors page does not release a session parked on that server

A session-less flow has no session to resume, so it does not run McpOauthResumeService at all — not even for a session that is failed with oauth_required on the very server you just authorized. The credential is stored and every future spawn picks it up, but the parked session stays parked until you click Authorize on its own banner (which then takes the already-have-a-credential branch: re-inject, clear the runtime needs-auth cache, resume).

A server that fails before it connects is listed as pending, not omitted

mcp_servers_status is built by McpLogPollerService from the per-server log directories Claude Code creates under ~/.cache/claude-cli-nodejs/<project>/mcp-logs-<name>/. A server that never gets far enough to create a log directory (e.g. an OAuth-blocked streamable-http server stripped from the launch) produces no key of its own.

McpStatusPersisting seeds a pending placeholder for every server in session.all_mcp_servers that the detector said nothing about, so such a server is at least listed. It used to be skipped outright. The session views already read an absent key as pending, but the JSON consumers do not — the REST API and the get_session MCP tool hand back custom_metadata verbatim, so a broken server simply was not in it, and absent there reads as “not configured” when the truth was “configured and broken”. The placeholder is written only when the key is absent, so it is a floor and never a correction: a real status, from this poll or any earlier one, is never overwritten by it. pending is still not failed — the log directory is the only evidence there is, and it does not exist — but the server no longer vanishes (#196).

One consumer treats it specially: McpServerBackfill#detect_lost_mcp_servers skips pending entries when it looks for servers a regenerated config dropped, since a placeholder is the absence of evidence rather than a server the session ever connected to.

The detector is not the only thing that applies that floor. Seeding it per poll leaves the key absent for as long as no poll has reached McpStatusPersisting — the whole clone-and-prepare phase of every turn, and forever on a turn whose process died before it wrote a transcript, since TranscriptPollerService returns early when there is no transcript file to read. Worse, resume used to delete the key outright, so a session that had been reporting connected for days went back to having no key at all on its next turn (#465). Two paths that run before any detector can therefore write the key themselves, and they are not the same operation:

Where	What it does
AgentSessionJob, immediately before the spawn	Calls Session#seed_mcp_servers_status_floor! — literally the floor above, pending only where no entry exists, so a status carried over from an earlier turn survives. It runs once air prepare has, so auto-injected servers are in all_mcp_servers and get an entry too.
SessionStateMachine#clear_stale_mcp_failure_metadata	Resets every entry to pending on each resume. This one deliberately overwrites a real status: a connected belongs to the process that just exited, not to the one about to start.

Both skip the write when what is stored already equals what they would write — the job’s floor when every server has an entry, the resume when every entry is already pending — so neither adds an UPDATE or a session-card broadcast to a steady-state turn. The resume spans the union of all_mcp_servers and the names already in the hash, so a soft-failing catalog read (which makes plugin_mcp_servers return []) cannot empty the reset and delete the key.

The runtime credential stores are host-global and shared across sessions

Claude Code reads ~/.claude/.credentials.json and its negative-auth cache ~/.claude/mcp-needs-auth-cache.json, both keyed by HOME, not by session — so every session on a worker shares them. Two consequences the credential writer handles: (1) a plain read-modify-write of .credentials.json races when two spawns overlap (last writer wins, silently dropping the other’s freshly-authorized token), so ClaudeMcpCredentialWriter serializes the read and the write under a flock on a sibling lock file — the same ClaudeCredentialStore lock the account writer takes for the claudeAiOauth block of that file, since an account rotation is a third racer (one credential file, three writers); (2) one session’s auth failure poisons the needs-auth cache for every later session — the entry makes Claude Code skip the connection outright, so a freshly-injected token stays invisible until the entry is removed. Injecting credentials and completing an authorize both clear the relevant cache entries. Codex re-reads its store on every connection and keeps no such cache, so its writer’s clear_needs_auth_cache is a no-op.