Threat Model — infrabroker¶
What this system defends, against whom, and — explicitly — what it does not cover. For how the mechanisms work see ARCHITECTURE.md; to report a vulnerability see SECURITY.md.
Premise¶
An AI agent needs to run commands on Linux hosts over SSH. The naive approach — hand the agent a static SSH key — fails because the key is exfiltratable (prompt injection, memory dump, a leaked tool log) and, once stolen, is valid until manually revoked. infrabroker removes the long-lived credential from the agent's reach: the agent receives only command output, never key material, and every operation uses a fresh, narrowly-scoped, minutes-long certificate.
The design defends two distinct threats:
- Credential theft — an attacker who reads the agent's memory/logs/traffic should gain nothing reusable.
- A compromised agent — an agent under prompt injection should not be able to run arbitrary commands, only those the operator's policy permits.
The first is fully addressed. The second is addressed for one-shot execution and partially for sessions — see the gaps below.
Assets¶
| Asset | Why it matters |
|---|---|
| CA private key | Signs every SSH certificate. Whoever holds it can mint access to any managed host. The crown jewel. |
| Ephemeral key pairs | One per operation, in broker memory only. Short-lived; their value is bounded by the cert TTL. |
| Audit log integrity | The forensic record. Tampering would hide abuse. |
| Host access | The ultimate target: shell on production Linux hosts. |
Policy & RBAC config (signer.json) |
Defines who may reach what. Its integrity equals the access boundary. |
Actors & trust levels¶
| Actor | Trust | Notes |
|---|---|---|
| AI model | Untrusted | Assumed subject to prompt injection. Sees only output; never holds credentials. |
Broker (infrabroker serve-mcp / serve-mcp-http / serve-http) |
Semi-trusted — may be compromised | Holds ephemeral private keys transiently. Never holds the CA key. Authenticates to the signer with its own mTLS CN. |
Control plane (control-plane) |
Semi-trusted (PEP) | Orchestrates approval and behavior guardrails. No CA key. Trusted by the signer only for on_behalf_of/approved if its CN is in trusted_forwarders. |
Signer (signer) |
Trusted | Sole custodian of the CA key. Authoritative for policy, RBAC, and the approval gate. Kept deliberately minimal; holds in-process policy plus grants/freezes/rate-limit state (optional state_db for durability) — not multi-replica-safe (see gap #5 / HA.md). |
| Operator | Trusted | Edits signer.json, approves out-of-band, holds approver/reload certs. |
Remote host sshd |
Trusted endpoint | Enforces force-command, source-address, principals, and sudoers — the last line of defense. |
| Log shipper (Vector, Fluent Bit, … — optional sidecar) | Untrusted reader | Third-party process with outbound network and the WORM/SIEM credentials. It only needs to read the audit files (root + BindReadOnlyPaths=, see OPERATIONS.md); it must never run as a service user, which would hand it that service's mTLS key, audit seed and (signer) policy write access. The exported copy is verified with audit verify, so a compromised shipper can drop or delay records, not forge them. |
The central design choice follows from this table: keep the CA key in the smallest, most-trusted component (the signer) and let everything else operate without it. A compromised broker or control plane cannot forge certificates.
Trust boundaries & guarantees¶
Model → broker¶
- The model never receives key material — only
stdout/stderr/exit_code. - stdio: isolation is the OS process (the MCP client launches the broker). Where the model's code runs — and what it can reach from there — is not this document's layer: it belongs to a sandbox such as ASP, whose microVM and deny-by-default egress compose with the broker (SANDBOX.md).
- HTTP: OIDC bearer token, validated locally against the issuer JWKS
(signature,
iss,aud,exp, andiatwhen a max age is set). Fail-closed (v1.11.2): a missing groups claim (whengroups_claimis configured) or a missingiat(whenmax_token_age_seconds > 0) rejects the token, so a misconfiguration cannot silently disable per-user RBAC. A forged or otherwise unverifiable bearer costs at most one JWKS fetch to the IdP, and those fetches are bounded by a local throttle (burst 5, then one per 10 s), so an unauthenticated client cannot drive the IdP fetch rate; tokens signed by an already-cached key keep verifying locally while the throttle is exhausted (#448).
Broker → signer (and via control plane)¶
- mTLS with TLS 1.3 minimum. The caller identity is the client-cert CN — not assertable by the broker in the request body.
- The broker sends an intent, not constraints. The signer derives every certificate constraint from policy; the broker cannot widen its own grant.
- Per-group RBAC (broker CN →
allowed_groups) and optional per-hostallowed_callersboth gate access; a broker must pass both. - Impersonation is unforgeable:
on_behalf_ofandapprovedare honored only when the mTLS CN is intrusted_forwarders(the control plane).
Signer → host¶
- One-shot: the command is baked into the cert's
force-commandby the CA key. sshd enforces it; the broker cannot alter it. This is the strongest guarantee in the system — it survives a fully compromised broker. - Scope pinning:
source-address(bastion egress IP on jump chains),ValidPrincipals, and a minutes-long TTL bound where, as whom, and how long a cert is usable. No agent/X11 forwarding extensions. - Command policy (allowlist/denylist/require-approval, with POSIX-sh
shell_parseAST checking on by default so chained/compound commands are checked stage-by-stage, and every shell construct outside a small accepted surface — simple commands, pipes,&&/||/;, fd→fd redirects, grouping,if/while— rejected fail-closed) restricts what one-shot command may run, and newlines are rejected so extra lines cannot be smuggled past the regexes.
Approval & audit¶
- Approval gate is authoritative and unavoidable: the signer issues no cert
for a
require_approvalcommand unlessapprovedarrives from a trusted forwarder. A direct broker cannot self-approve, and the originator of a request cannot decide its own approval (four-eyes, even if its CN is an approver). Each approval is consumed once. - Approval-bridge trust (
cmd/approval-bridge, #120). The chat bridge decides with a single approver CN, so its residual dilutions are, by design: (1) who may approve moves partly from the mTLSapproval.callersallowlist to chat channel membership run by an external SaaS — anyone who can click a button in that channel can approve; (2) approver attribution is bridge-asserted — the signed audit records the bridge's approver CN; the platform user id who clicked is written only to the bridge's own process log (operational, not tamper-evident), so the decision is not tied to a cryptographically verified approver; (3) the control plane's four-eyes guard compares the request's originator against the bridge's approver CN, which never collides — so on its own it cannot stop the human who both originated the request and clicks Approve in chat. The bridge closes that gap when an identity map is configured (approval-bridge --identity-map, a{platform_user_id: end_user_identity}file): it refuses an approval whose clicker maps to the request's originating end user (#214). The guard can only work when the request carries anend_user: a request whose broker is not in the control plane'strusted_forwardersreaches the bridge with its asserted end user stripped (#451), so the map cannot attribute the originator and the guard is inert for it — the card saysuser (asserted by an untrusted broker, dropped)so the approver sees why; add such brokers totrusted_forwardersto restore it. Without the map that guard is off (the prior residual), and per-human approver certs collapse into the single bridge CN. The control plane's consumed-once guard is unchanged. For a single operator, #118's in-chat elicitation avoids the bridge entirely; for stronger attribution, approve viabroker-ctl/ the web UI (per-human mTLS certs). The approver's chat card renders every broker-supplied field (command, host, identities) as plain text, so a crafted command cannot inject a clickable link or formatting into the approval prompt (#239 — the bridge analogue of the Teams Adaptive Card escaping, #174). Every approver surface (the web UI, the Slack and Teams cards, the log notifier) also renders Unicode format characters (bidi overrides/isolates, LRM/RLM, ZWJ, BOM), C1 controls and U+2028/2029 in those fields as visible\u{XXXX}escapes (#830), since the signer rejects only CR/LF and a bidi-aware renderer would otherwise display a destructive stage reordered as a trailing comment; presentation only — the stored request, the signer input and the JSON API keep the original text. - Audit log is append-only, SHA-256 hash-chained, and Ed25519-signed per
entry; modifying a record, reordering records, or deleting a record in the
middle of the chain is detectable by replaying it. The chain stays continuous
across log rotation — each rotated-to file's first entry links to the
previous file's last hash — so dropping an earliest or interior rotated
segment is detectable with
broker-ctl audit verify --all, which verifies the whole segment set and the cross-file linkage. Note that single-fileverifyaccepts the first entry'sprev_hashas an unchecked seed, so cross-segment integrity requires--all(v1.13.0). Not detectable: tail truncation — dropping the newest records, or the newest rotated segment(s) together with the active file, leaves a shorter but internally valid chain, and truncating the active file and restarting does not re-anchor to genesis either (the chain is re-seeded from the newest rotated segment, #279), soverify --all --keyreports no error. Anyone with write access to the log directory (the seed is not needed) can do this; see gap #14. Three logs (signer, broker, sshd) correlate by certserial.
Defense in depth (one-shot)¶
A single malicious one-shot command must pass, in order:
- Frontend auth (process / OIDC token).
- Broker→signer mTLS + group RBAC +
allowed_callers. - Per-user RBAC (OIDC groups ∩ host groups), if applicable.
- Command policy (allow/deny,
shell_parse, newline rejection). - Approval gate (if
require_approval). - Behavior guardrails (if the control plane is in
enforce). - On the host:
force-command,source-address, principal, sudoers.
Layers 4–7 are what make this more than a credential vault.
Process isolation on a colocated host. The reference deployment
(deploy/) runs each service as its own system user with a per-service
PKI subdirectory and per-service config group. A compromised broker frontend
therefore cannot read the signer's CA key (pem custody), policy, grant
state, audit seed, or mTLS key — nor impersonate the signer, the control
plane, or the admin CLI (whose material is root-only). Writes were already
contained by the systemd sandbox; the user split contains reads. The
installer, which runs as root, never chowns or chmods by path inside the
signer-owned state directory and refuses a symlinked config, so a compromised
signer cannot plant a link there to take over a root file on the next upgrade
(run it with the units stopped, per deploy/README.md § Upgrades). Running the
signer on a separate host remains the stronger posture.
Explicit non-goals & gaps¶
These are deliberate limits, not oversights. Naming them is the point of this document — they define where additional controls (or a different tool) are needed.
1. Session command firewall is broker-enforced, not host-enforced¶
force-command only applies to one-shot. In a session the cert authenticates
the connection and commands flow as separate channels; the host does not see the
signer's per-command decision. The broker preflights every ssh_session_exec
against the current signer policy, so policy reloads affect sessions that were
already open. The preflight revalidates target access, bastion access, end-user
groups, sudo, sudo_user, PTY, and the physical SSH chain
(addr/user/host_key/jump); if the host route changed since the session
was opened, the broker rejects the next command and the caller must open a fresh
session. On command-policy hosts,
mode=exec commands are also checked before execution, and shell/pty session
commands are rejected because stateful command streams are not independently
verifiable. This protects against a compromised/prompt-injected model using the
normal broker tool path. It does not survive a compromised broker that obtains
a session cert and skips the preflight. On hosts without a command policy, the
command text itself is not restricted by infrabroker; it can run anything the
host's sudoers/principal allow.
- Mitigation today: prefer ssh_execute on sensitive hosts when you need the
host-enforced force-command guarantee; use mode=exec sessions only when
connection reuse matters and broker-side preflight is an acceptable control.
Keep source-address + principal + restrictive sudoers. OpenSSH validates
the certificate only at authentication, so an established connection would
itself outlive expiry; the broker reaper enforces the bound and closes the
session at certificate expiry, after session_idle_seconds of inactivity,
or after session_max_seconds of total lifetime — whichever comes first
(#124, #225). A session's exposure is therefore capped at the certificate
TTL (≤ max_ttl_seconds); set session_max_seconds to tighten it further.
- Mitigation (opt-in per host) — "sealed exec" (#144): makes session-exec
filtering host-enforced, the way one-shot already is. Set "sealed_exec": true
on a host in signer.json and configure envelope_key (a dedicated Ed25519
seed — deliberately not the SSH CA). That host's session cert then carries
force-command=infrabroker-shim, and at the existing per-command preflight
(already a signer round-trip, so no new hot path) the signer signs an envelope
{nonce, host, command, expiry}. The broker sends the envelope as the SSH
channel command; the static infrabroker-shim on the host verifies it against a
pinned envelope public key (/etc/infrabroker/envelope.pub, logged by the signer
at startup), checks it is bound to this host, enforces the expiry, claims the
nonce once, and refuses to execute anything unsigned. Per-command authorization
therefore survives a fully compromised broker — structurally out of reach for
end-to-end-tunnel products, which keep no signer in the command path. The
elevation prefix travels inside the signed envelope, so the shim (not the
broker) applies sudo.
On a sealed host every certificate is pinned — either to the exact one-shot
command or to the shim — including bastion-role certs, so a compromised broker
cannot ask for a role whose cert carries no force-command and get a free shell.
The host name is carried in the force-command itself (infrabroker-shim
<host>): the shim takes its identity from the signer-signed certificate, which a
compromised broker cannot forge. That binding is load-bearing — every sealed host
pins the same envelope key, so without it an envelope minted for a permissive
host would replay verbatim on a restrictive one. ProxyJump is unaffected
(direct-tcpip channels never run a force-command).
Constraints:
- Sealed hosts are restricted to mode=exec; shell/pty are not
envelope-verifiable and are rejected unconditionally.
- Replay is bounded by the nonce (claimed once in a host-side store, which must
exist and be writable by the SSH account) plus a short expiry; an envelope
whose expiry is more than MaxTTL ahead is refused, so a skewed host clock
cannot stretch the window.
- Commands run via /bin/sh -c, not the account's login shell — a command
relying on a login-shell dialect (a bashism) may need an explicit interpreter.
- Turning sealed_exec on does not seal sessions that are already open:
OpenSSH validates a certificate only at authentication, so pre-existing
sessions keep their unpinned cert until they are closed. Close them (or use the
kill switch) after flipping the flag.
- Off by default, and remote-topology only: it exists to survive broker
compromise, which presupposes broker != signer.
- An sshd_config ForceCommand on a sealed host defeats it silently.
OpenSSH prefers the configured command over the certificate's
(options.adm_forced_command is checked before auth_opts->force_command in
session.c's do_exec), so the shim never runs and the signed envelope
arrives in $SSH_ORIGINAL_COMMAND of whatever was configured. This is host
configuration the signer cannot see or attest; install-shim.sh --check
probes it with sshd -T, and LogLevel VERBOSE distinguishes
forced-command (key-option) from (config) in the sshd log.
- So does a startup file the SSH account can write. The force-command is
the bare name infrabroker-shim, run as <login shell> -c '…', and that
shell sources per-account files first: bash reads ~/.bashrc when started
by sshd (SSH_CLIENT set on the Debian/Ubuntu and Fedora/RHEL builds; stdin
a socket upstream), zsh always reads ~/.zshenv, fish config.fish,
csh/tcsh ~/.cshrc. A shell function named infrabroker-shim defined there
(functions win over PATH) — or an exec "$SSH_ORIGINAL_COMMAND" — runs
instead of the verifier, so one write by the account (pre-sealing access, an
allowed file-writing command, file transfer) turns every later certificate
into an unverified one. Mitigation is host configuration: /bin/sh as the
account's shell where it is dash/ash (bash invoked as sh, the RHEL
/bin/sh, reads no ~/.bashrc either), or a home, rc files and ~/.ssh
owned by root and not writable by the account. Likewise
AuthorizedKeysFile none for broker-managed accounts (in a Match User
block): a key the account adds to its own authorized_keys logs in with no
certificate and so no force-command. install-shim.sh --check fails on a
startup file that mentions infrabroker-shim and warns (without failing) on
account-controlled startup files, a present authorized_keys, or an
AuthorizedKeysFile other than none (docs/OPERATIONS.md, Harden the
sealed account).
- Permanent maintenance tail: multi-arch shim releases and a version-skew
matrix. Envelope-key rotation is no longer part of that tail: the pinned
key file accepts more than one key, so a host that pins the outgoing and the
incoming key together stays up while the signer is switched over (runbook in
docs/OPERATIONS.md § 2.2). The skew rule that follows from it: a shim older
than v3.1.0 reads the file as a single key and fails closed on two lines, so
every sealed host must be upgraded before a second key is appended.
Deployment (v3.1.0): deploy/install-shim.sh places the shim binary, the
pinned public key and the nonce store on a sealed host, and --check re-verifies
them. It is a separate script from install.sh, which still targets only the
service hosts. What it cannot do for you is decide which SSH accounts a sealed
host exposes — you name them, and they gain write access to the shared nonce
store (1770 root:infrabroker-shim: each account can claim a nonce, none can
delete another's claim).
- Composition note (v1.14.0): a host's effective firewall is the composition
of its inline command_policy and the policies of all its groups (additive:
deny wins, allow is a union). This makes group membership security-relevant:
assigning a host to a group can widen its allow-set, not only narrow it. Treat
group_command_policies as part of the firewall config, keep allowlists minimal,
and use the _default group (applies to every host) for global denylist
guardrails (e.g. ^rm, ^reboot). A host left out of every allowlist group but
carrying a _default denylist is default-allow except for the denied patterns —
use an allowlist group for true least-privilege.
2. Behavior guardrails are detection, not containment¶
The guardrail subject is the authenticated broker CN (the mTLS client
certificate). For guardrail keying, the client-supplied end_user only
qualifies the subject (<broker CN>:<end_user>) when the broker CN is listed
in the control plane's trusted_forwarders — i.e. a broker the operator
trusts to authenticate end users (e.g. via OIDC). For any other CN the control
plane ignores the unauthenticated end_user when keying baselines and rate
limits, so a client cannot reset them by rotating it (fixed in v1.12.6);
the residual guardrail gap is a trusted forwarder that is itself compromised
rotating the end_user half of its own subject.
In enforce, a novel host/command is not learned while it is pending approval;
retrying the same unapproved anomaly remains anomalous. Behavior remains a
detection layer, not the authoritative containment boundary: the hard controls
are the signer-side policy and approval gate, which a broker cannot bypass.
Note the scope of that gating: it protects the guardrail subject only. By
default the signer accepts end_user / end_user_groups verbatim from every
authenticated caller CN — its trusted_forwarders list gates only approved
and on_behalf_of, not identity. End-user identity is verified where the
OIDC token lives (the infrabroker serve-mcp-http frontend); by the time an intent
reaches the signer it is a caller-asserted label — unless the caller opts into
signer-side re-validation (see the mitigation below). Concretely, for any CN in
callers without require_verified_end_user:
- The asserted
end_useris stamped verbatim (charset-sanitised, never authenticated) into the certificateKeyID(user=) and the signed audit trail: a compromised or malicious broker can attribute its actions to an arbitrary end user. The audit log proves which CN asserted the identity, not that the identity is real. The assertedend_user_context(the frontend'scontext_claim, e.g. an ASPsandbox_id;ctx=in the KeyID) has exactly the same posture: attribution a non-gated broker can forge, and the same mitigation below re-derives it from the verified token. - Asserted
end_user_groupsselect the Kubernetessa_bindingand, when omitted (null), skip the per-user host-group narrowing. They can never widen access beyond the caller CN's own allowlist, but with group-differentiatedsa_bindingsthe choice of ServiceAccount rests on a claim the signer does not verify. - A grant or waiver scoped only by
end_user(itscallerfield left empty) matches any caller CN asserting that name.
The identity boundary at the signer is therefore the mTLS caller CN;
end_user is attribution within it, trustworthy exactly as far as the
asserting broker is. Keep the callers table minimal, scope grants by
caller (not only end_user), and treat group-differentiated sa_bindings
as trusting every CN authorised for that cluster.
- Mitigation (opt-in, #143): signer-side re-validation moves end-user
authenticity inside the trust boundary. Set "require_verified_end_user": true
on a caller CN and configure an end_user_oidc block (issuer/audience/claims
mirroring the frontend's oauth): the frontend forwards the raw bearer over the
mTLS channel and the signer re-validates it against the issuer JWKS, deriving
end_user/end_user_groups from the verified JWT instead of the broker's
assertion. Fail-closed (a gated caller with a missing/invalid token, or no
issuer configured, is denied) and off by default, so the caller-asserted
behaviour above is what applies until a CN opts in. The forwarded token is a
secret held only over the wire and in memory — never logged, audited, or
persisted (the control plane strips it before writing a pending approval), so a
control-plane restart mid-approval, or a token aging past max_token_age before
a human decides, makes that one approval fail closed and be re-requested.
3. No certificate revocation (KRL)¶
Mitigation is the short TTL (minutes). A certificate leaked within its validity window is usable until it expires; there is no way to cut that individual issued cert short.
The kill switch (#117) addresses the exposures a KRL would, at the policy
layer rather than by distributing revocation lists to every sshd. A frozen
subject (broker CN, end_user, session_id, or certificate serial,
POST /v1/freeze, reload_callers only) is enforced at two layers, and
which layer applies depends on the subject kind:
- signer-side deny (the hard guarantee) — a frozen caller CN is denied
every new certificate at POST /v1/sign and all connectivity at
GET /v1/hosts, so one-shot, session-open and each session-exec preflight
fail immediately. A frozen end_user is denied at POST /v1/sign only:
/v1/hosts carries no end user, so the broker still lists hosts for them
and the denial lands on the next sign (audited frozen_denied). A frozen
session_id or serial is not checked by the signer at all — it
names something already issued, which the signer cannot map to a new
request. Both caller and end_user freezes also revoke the subject's
runtime grants and approve-and-learn waivers.
- broker-side kill (best-effort containment) — every freeze, of every
kind, is streamed to brokers via GET /v1/revocations, which every broker polls
(revocation_poll_seconds, default 10s). A matching session is force-closed
including one with a command in flight — the kill overrides the reaper's
never-close-a-busy-session rule — so a compromised subject is bound to one
poll interval rather than the session lifetime. session_id/serial target a
specific session/cert; a frozen end_user closes all of that user's sessions;
a frozen broker caller CN closes every session that broker holds.
The freeze set is fail-closed persistent (state_db): unlike a grant, a freeze
that failed to load would fail open, so the signer refuses to start on a load
error rather than silently un-freezing a blocked subject. A freeze only survives
a restart with state_db, so since v2.0.0 POST /v1/freeze refuses a freeze
when state_db is unset unless the caller explicitly accepts a memory-only
freeze (allow_volatile / broker-ctl freeze --volatile) — the fail-open of a
silently-volatile freeze is opt-in, not the default.
- Residual: an already-issued one-shot cert stays valid for its
remaining window (
<= max_ttl, minutes) — a freeze denies the next sign and ends sessions, but does not recall a one-shot cert already handed to a live connection. A truly compromised broker can also ignore the revocation poll and skip its own session kills (though it still gets no new certificates); the signer-side deny is the hard guarantee, the broker kill is best-effort containment. An OpenSSH KRL by serial (roadmap) would close the one-shot gap.
4. Rate limiting on the signer is opt-in¶
The signer supports a per-CN token-bucket rate limit on POST /v1/sign
(sign_rate_limit_per_min, hot-reloadable): keyed on the authenticated mTLS
peer CN, enforced before body parsing, excess requests get 429 with a
Retry-After hint, and rejections are deliberately not audited so the
tamper-evident log cannot become the flooding amplifier. The same per-CN
bucket caps the audited authorisation denials on the admin endpoints
(/v1/reload, policy read/mutation, grants): past the budget a non-admin CN
gets 429 and no *-denied record. The frozen-subject denials on
GET /v1/hosts / GET /v1/clusters have their own per-frozen-subject audit
budget (same limit, separate bucket so a frozen broker behind the control plane
cannot drain its sign budget): always 403, but only limit frozen_denied
records per minute are written. Neither lets a client force one
signed+fsynced record per request. The residual gap is
that the cap is opt-in (0/absent = disabled, backward compatible) — set it in
production. The control plane additionally applies its own per-subject
behavioral rate limit on the forwarded path.
5. In-memory state → single instance¶
Sessions, approvals, grants, freezes, behavior baselines and rate-limit buckets
live in process memory — in all three processes, the signer included. Running
multiple broker, control-plane or signer replicas would split this state.
Horizontal scaling requires externalizing it (e.g. Redis with TTL).
- Decided degradation for the budget layers (#295, #297): the per-CN
sign_rate_limit_per_min and the control plane's per-subject
rate_limit_per_min are per process, so N replicas admit up to N× the
configured cap, and behaviour baselines split-brain (producing extra
escalations, never missed ones). This is accepted and documented rather than
engineered away: budgets are a detection layer, and containment remains the
signer's command policy and the approval gate, which are config-derived and so
identical on every replica. See HA.md for the full reasoning, the
sizing guidance, and the custody constraint that agent (ssh-agent) custody is
host-local and does not replicate the way akv does.
- Mitigation (restart survival, not multi-instance): the opt-in state_db
(SQLite, write-through) persists the signer's runtime grants/waivers, the
kill-switch freeze set, and the control plane's approval registry across
restarts. Grants/waivers fail safe if dropped (they only widen), but a dropped
freeze fails open — so without state_db a freeze is refused unless explicitly
opted into as volatile (gap #3). The in-memory state remains the only state
consulted on the decision path; live SSH sessions and behaviour baselines are
intentionally not persisted (a TCP connection cannot be resurrected; the
baseline re-learns).
- Residual risk (crash window): an approval is marked consumed with a
best-effort write after the certificate is issued. A crash (or a state-db
write failure, counted by statedb_errors_total) in that window re-exposes
the approval as consumable once more after the restart — bounded by the
approval TTL and the certificate TTL. Grant revocation takes the opposite
trade: the db delete is mandatory, so a revoked grant can never resurrect.
6. callers is default-deny once RBAC is in use¶
Since v2.0.0 a non-empty callers table is authoritative and default-deny: a
broker CN absent from it sees and signs no hosts. Forgetting to list a CN now
fails closed. An unlisted CN inherits the reserved _default entry if present,
else the zero policy (no groups) — either way, no hosts.
- Documented opt-out (unrestricted): omitting callers entirely (an empty
table) configures no group RBAC and leaves every authenticated broker CN able
to request all hosts — the right default only for a single-broker deployment.
broker-ctl doctor --security WARNs when no callers table is present.
- Widening: to grant unlisted CNs a baseline instead of denying them, add a
reserved "_default": {"allowed_groups": [...]} entry with groups
(broker-ctl callers add --name _default --groups "..."); doctor WARNs on a
_default that grants groups, since it widens past the default-deny.
- Defence in depth: per-host allowed_callers pins sensitive hosts to
specific CNs regardless of the group RBAC.
- Control-plane role separation: the control plane separates the broker role
from the approver role on the signing path (/v1/sign, /v1/hosts,
/v1/sign/result). With no sign_callers list a CN in approval.callers is
denied the sign path (an approver is not a broker — secure by default); a
non-empty sign_callers is an exact allowlist. An empty or control-character
client-certificate CN is rejected (fail-closed) rather than treated as an
unlisted, default-open identity.
7. CA key custody depends on deployment¶
Local/lab mode loads the CA key from a PEM file into process memory (a runtime
[WARN] flags this). Production should use a non-PEM backend behind the custody
seam: akv (Azure Key Vault; RSA/EC only) or agent — the CA private
key in a running ssh-agent backed by a YubiKey PIV slot / SoftHSM / TPM
(ssh-add -s), with no cgo in the signer and support for Ed25519 CA keys
(which AKV lacks). The private key never leaves the backend. Using PEM in
production is an operator error the code warns about but cannot prevent.
Note the two are not interchangeable if you ever replicate the signer: akv is a
network service each replica authenticates to independently, while agent is a
host-local unix socket — it does not compose with N replicas without one
token per replica (separate CA keys, a wider trust surface) or socket forwarding
that would defeat the custody boundary. See HA.md (#297).
8. Secrets in commands: redaction is default-on and best-effort¶
A command is written to the broker and signer audit logs and, for shell/pty
sessions, to the ASCIIcast recording; the control plane additionally sends it
in approval notifications (log/webhook/Teams). A credential passed inline —
mysql -psecret, PGPASSWORD=… pg_dump, curl -H "Authorization: Bearer …" —
would otherwise persist in plaintext in every one of those sinks.
- Mitigation: the redact config block (signer, broker, control plane)
masks secrets at every persistent/outbound sink — audit log free-text
fields, session recordings, and the approval notification payload — using
built-in patterns plus operator-defined RE2 rules, replacing the secret with
[REDACTED:<rule>] before the audit entry is signed (verification is
unaffected; the original is irrecoverable). The built-in rules are ON by
default — no redact block selects them (#400: a compromised broker can
plant broker-chosen fragments in the Err/Warning fields, so the persistent
sink must scrub them without waiting for an operator to opt in); the block
extends the defaults with operator patterns or disables the built-ins
(disable_defaults). The approval-bridge reads the
original command from the control plane's /v1/approvals (an mTLS approver
like any other) but masks it with the built-in default patterns before
presenting it on the chat platform — an off-host sink of the same class as the
webhook/Teams notifier — so a chat channel is never sent a secret the Teams
notifier would have masked. Redaction never touches the decision path: the
signer, the certificate force-command, and the mTLS approval UI (broker-ctl /
the web UI) see the original command. Because the bridge card is itself a place
to decide, a card whose command the bridge masked or truncated (#666) is
deny-only (#829): the requester chooses what text a pattern masks, so an
Approve beside masked text would approve a command the human never saw. The
card offers Deny plus a notice to review the original in the approval UI, and
the bridge refuses an Approve for any approval it did not post with an Approve
control (a forged or stale interactive payload).
- Residual risk: pattern matching is best-effort, not DLP (see
SECURITY.md) — an unanticipated
secret format survives, and output recorded in .cast files arrives in
arbitrary chunks that can split a secret across two events. The masked span is
pattern-defined, and a requester can deliberately plant text a pattern matches,
so masking can also hide part of a command from the audit log (the chat card
being deny-only covers only the approval decision). Prefer
credential-free invocations (env files on the host, ~/.pgpass, secret
managers) and keep treating audit logs / recordings as sensitive at rest
(0600, restricted directories).
9. Audit failure is fail-closed by default¶
If writing an audit entry fails (disk full, I/O error), the audited action is
denied — "no signed audit record, no action". Since v2.0.0 this is the
default (audit_fail_mode: "closed") on both the signer and the broker:
- Signer (the load-bearing gate): a failed issuance audit means the
certificate/bound token is never written to the response, so no downstream
action can proceed. Because credentials are per-operation ephemeral certs with
a force-command, gating issuance transitively covers one-shot exec, sessions,
and file transfers. The sign request returns 500 "audit unavailable".
- Broker (defence in depth): an action-completion record that cannot be
written (ssh_execute, session open / ssh_session_exec, file transfer, k8s
action) causes the result to be withheld (500 "audit unavailable"). The
remote side effect of a post-execution audit failure already happened — the
documented residual — but the output is never returned without a durable trace.
- Control plane (approval path): the same default governs the two
approval-critical records. An allow decision whose approval-decision-allow
record cannot be written is answered 503 "audit log unavailable" to the
approver: the decision is already recorded in memory (a retry answers 409),
but no credential is released, because the grant path gates on its own
approval-granted record and withholds the certificate/token with the same
503 until the log is writable again; the approval stays collectable.
- Documented opt-out (availability over the guarantee): set
audit_fail_mode: "open" to restore the pre-2.0 behaviour (log the error and
proceed). The availability trade of the closed default is honest: a full or
unwritable audit disk stops actions until it is resolved — the runbook note
in Operations says what to check when actions fail with
"audit unavailable".
- Alerting: audit_append_failures_total counts every write failure and
audit_blocked_total counts actions denied because of one; both are exposed on
monitor_listen (/metrics). The signer 429 rate-limit rejections remain
deliberately un-audited (the log must not amplify a flood), so they are never
affected by this mode.
10. Kubernetes: token grants the SA's RBAC, not a single call¶
The Kubernetes target (v1.34.0) is a credential-broker: for an authorised
action the signer mints a bound ServiceAccount token and the broker runs the
one API call with it. Two structural differences from SSH follow, by design:
- No inevadible per-call firewall. SSH bakes a force-command into the
certificate and sshd enforces it, so the credential does exactly one thing.
A Kubernetes token instead carries the whole RBAC of its ServiceAccount
for its lifetime (600–900s), so within that window it can do anything that SA
may do — not only the approved action. Granularity is therefore the SA's
native RBAC (layer B) refined by the broker's action policy (layer A), not
"only this exact object". Mitigation: scope each agent ServiceAccount to
least privilege (the layer-A default-deny policy bounds what the broker will
request, but layer B is what the token actually grants); keep the bound
TTL at the 600s floor; do not add secrets to an allow rule unless required.
- The signer holds a standing cluster credential. The per-cluster minter
token (token_file) is a long-lived credential — unlike the SSH CA, which
signs but never authenticates as a principal. Its RBAC is deliberately
minimal (only create on serviceaccounts/token for the bound SAs), so a
signer compromise yields token-minting for those SAs, not cluster-admin.
Mitigation: the minter SA's Role must grant nothing else; rotate the
token_file out-of-band (the signer re-reads it per mint).
The reused control plane keeps its guarantees: the action's canonical string is
recomputed by the signer and must match the structured request (so the approver
and the audit log see what runs), a k8s_apply manifest is never logged
verbatim (only its sha256), and require_approval, grants, and approve-and-learn
apply to k8s actions exactly as to shell commands.
- Corollary for k8s_apply — approval gates the target, not the payload.
Because the manifest never leaves the broker (only its sha256 is audited), an
approval for apply deployments prod/api authorizes an arbitrary manifest
spec for that object — image, privileges, replicas, env — that no human
reviewed. The API server binds the path to metadata.name/namespace, but
not the rest of the spec, so the approver sees only the verb/resource/
namespace/name, not what is applied. Mitigation: scope apply rules
narrowly (pin namespaces/names) and reserve them for trusted flows;
prefer require_approval on apply only where the target coordinates alone
are a sufficient gate. This is the deliberate cost of not shipping (possibly
secret-bearing) manifests through the control plane.
11. Dry-run is a policy oracle (by design)¶
ssh_execute / k8s_* calls with dry_run=true return the decision without
executing — including reason_code, the matched policy rule
(MatchedRule, e.g. deny:^rm) and Reason — so the model can branch on
why a request was denied. The flip side: any caller that can reach the
broker can enumerate allow/deny patterns and require_approval rules for a
host one probe at a time. Dry-runs are best-effort audited (dry_run
outcome, no fail-closed gate — nothing ran to withhold). Accepted residual:
treat the caller's identity as the authorization for seeing policy shape; do
not grant multi-tenant or untrusted agents broker access if the policy
grammar itself is sensitive, and remember allowlist policies are trivially
readable this way.
12. ssh_get_file on an enabled host is an arbitrary read (by design)¶
With allow_file_transfer=true on a host, ssh_get_file reads any file the
host's configured SSH user can open — /etc/shadow-adjacent service TLS
keys, ~/.aws/credentials, whatever — subject only to command_policy, and
the policy is a coarse switch here, not a filter: the transfer forms
(cat > 'path' / head -c N < 'path') use a file redirect, which
shell_parse (the default) rejects outright, so on a host with any
restricting policy — allowlist, denylist or require_approval-only — every
transfer is denied in enforce mode (audit mode runs it with a would-deny
warning) and no allow pattern changes that. Only a host whose policies all
set shell_parse: false can allowlist the transfer forms
(^cat > '…'$ / ^head -c …$), and there a denylist alone does not
constrain the read. A host with allow_file_transfer=true and no
command_policy reads anything.
Output goes straight into the model's context. Accepted residual: pin
the SSH user to the narrowest usable account and treat
allow_file_transfer=true as granting read access to everything that user
can read. (Likewise ssh_put_file is an arbitrary write wherever that user
may write.)
13. Out of scope entirely¶
- Confidentiality of command output beyond transport TLS (the model sees it by design).
- Compromise of the signer host or the operator's credentials (top of the trust chain — if the CA key host is owned, the model is moot).
- Supply-chain integrity of the Go dependencies.
- Network-level DoS below the application layer.
14. Audit-log tail truncation is not detectable locally¶
The audit chain is a hash chain with no anchor for its head: verify and
verify --all prove that what is on disk is internally consistent (no modified,
reordered or interior-deleted record, no dropped earliest/interior segment), but
cannot tell that the newest records are missing. Dropping the last N lines of
the active file, truncating the active file and restarting (re-seeded from the
newest rotated segment, #279, so the chain stays continuous), or removing the
newest rotated segment(s) together with the active file all verify clean.
- Who can do it: anyone with write access to the log directory; the Ed25519 seed is not required, because no record is forged.
- Mitigation (operational): ship the log off-host in near real time (the
shipper recipe in OPERATIONS.md, ideally to Object Lock/WORM)
and compare the last
seq/prev_hashof the shipped copy against the local log. A shipper can itself drop or delay records (see the Log-shipper row of the trust table), so the local file alone is never proof of completeness. - Future work (not a commitment): a head anchor stored outside the log directory.
Summary¶
| Threat | Status |
|---|---|
| Credential exfiltration from the agent | Mitigated — no reusable credential ever reaches the model |
| Compromised agent, one-shot commands | Mitigated — policy + force-command + approval, signer-authoritative |
| Compromised agent, sessions | Partial / host-enforced opt-in — every ssh_session_exec is broker-preflighted; shell/pty rejected once policy is active; sealed_exec + infrabroker-shim makes the per-command decision host-enforced on the hosts that opt in (gap #1) |
| Compromised broker forging access | Mitigated — no CA key; signer derives all constraints |
| Stolen cert reuse within TTL | Accepted risk — no revocation; bounded by minutes-long TTL |
| Compromised agent, Kubernetes actions | Partial — layer-A default-deny action policy + approval; layer-B is the SA's RBAC (a bound token grants the SA's whole RBAC for its TTL, not one call — gap #10) |
| Audit-log tail truncation by a writer of the log directory | Accepted residual — interior tampering is detected, newest-record loss is not; ship off-host and compare (gap #14) |
| Signer/operator compromise | Out of scope — trusted root |
The credential-custody story is strong and complete. The action-control story is
strong for one-shot and, by default, weaker for sessions because per-command
filtering is broker-enforced, not host-enforced. Gap #1 is closed for the hosts
that opt into sealed exec (sealed_exec + infrabroker-shim, #144): there the
per-command decision survives a compromised broker, the way one-shot always has.
Gap #3 (no KRL) remains the open item and the highest-value security investment.