Substrate architecture and data flows
-
-
-
-
-
+
+
+
+
+
-
+
-
- Kubernetes cluster
-
- ate-system namespace: control plane
-
- Kubernetes
-
- podcertificate-controller-system
-
- Worker node
-
- Worker Pod, worker namespace
-
- Sandbox: gVisor or microVM
-
- Sandbox: gVisor or microVM
+
+ Kubernetes cluster
+
+ ate-system namespace: control plane
+
+ Kubernetes
+
+ podcertificate-controller-system
+
+ Worker node
+
+ Worker Pod, worker namespace
+
+ Sandbox: gVisor or microVM
+
+ Sandbox: gVisor or microVM
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
- Application layer
- agent framework,
- end-user auth
+
+ Application layer
+ agent framework,
+ end-user auth
-
- External relying
- parties
+
+ External relying
+ parties
-
- External services
- and tools
+
+ External services
+ and tools
-
-
- Snapshot bucket
+
+
+ Snapshot bucket
-
-
- Sandbox assets
- (SandboxConfig)
+
+
+ Sandbox assets
+ (SandboxConfig)
-
- atenet-router
- ingress gateway
+
+ atenet-router
+ ingress gateway
-
- ate-api-server
- OpenFGA authz, identity minting,
- objectstore-plugin sidecar
+
+ ate-api-server
+ OpenFGA authz, identity minting,
+ objectstore-plugin sidecar
-
- ate-controller
+
+ ate-controller
-
- ate-idp-server
- OIDC discovery, JWKS
+
+ ate-idp-server
+ OIDC discovery, JWKS
-
-
- PostgreSQL
+
+
+ PostgreSQL
-
- k8s-credential-provider
- egress secrets
+
+ k8s-credential-provider
+ egress secrets
-
- atenet-egress
- egress gateway
+
+ atenet-egress
+ egress gateway
-
- kube-apiserver
+
+ kube-apiserver
-
- podcertcontroller
- pod, DNS, DB certs
+
+ podcertcontroller
+ pod, DNS, DB certs
-
- ateworker + atunnel
- ingress listener, egress redirect,
- DNS relay
+
+ ateworker + atunnel
+ ingress listener, egress redirect,
+ DNS relay
-
- atelet DaemonSet
- objectstore-plugin sidecar
+
+ atelet DaemonSet
+ objectstore-plugin sidecar
-
- Actor A
- untrusted code, no credentials
+
+ Actor A
+ untrusted code, no credentials
-
- Actor B
- untrusted code, no credentials
+
+ Actor B
+ untrusted code, no credentials
-
- Container image
- registry
+
+ Container image
+ registry
-
- A1
- create actor
-
- A2
- save record
-
- B1
- send request
-
- B2
- resume actor
-
- B3
- restore actor
-
- B4
- get snapshot
-
- B5
- start sandbox
-
- B6
- forward request
-
- B7
- deliver
-
- B8
- call a tool
-
- B9
- forward as this actor
-
- B10
- get credential
-
- B11
- call with credential
-
- A: create an actor
-
- B: send a request to an actor
-
- Supporting connection
-
- Sandbox: untrusted actor code
-
- Worker Pod: shared by many actors
-
- Node
-
- Control plane namespace
-
- Untrusted
-
- Node and worker components
-
- Control plane components
-
- Outside Substrate
+
+ A1
+ create actor
+
+ A2
+ save record
+
+ B1
+ send request
+
+ B2
+ resume actor
+
+ B3
+ restore actor
+
+ B4
+ get snapshot
+
+ B5
+ start sandbox
+
+ B6
+ forward request
+
+ B7
+ deliver
+
+ B8
+ call a tool
+
+ B9
+ forward as this actor
+
+ B10
+ get credential
+
+ B11
+ call with credential
+
+ A: create an actor
+
+ B: send a request to an actor
+
+ Supporting connection
+
+ Sandbox: untrusted actor code
+
+ Worker Pod: shared by many actors
+
+ Node
+
+ Control plane namespace
+
+ Untrusted
+
+ Node and worker components
+
+ Control plane components
+
+ Outside Substrate
diff --git a/docs/assets/actor-template-diagram.svg b/docs/assets/actor-template-diagram.svg
index 179e966453..8f56a7fe7f 100644
--- a/docs/assets/actor-template-diagram.svg
+++ b/docs/assets/actor-template-diagram.svg
@@ -2,131 +2,131 @@
Substrate actor template creation
-
-
-
-
-
+
+
+
+
+
-
+
-
- Kubernetes cluster
-
- ate-system namespace: control plane
-
- Worker node
-
- Worker Pod, worker namespace
-
- Sandbox: gVisor or microVM
+
+ Kubernetes cluster
+
+ ate-system namespace: control plane
+
+ Worker node
+
+ Worker Pod, worker namespace
+
+ Sandbox: gVisor or microVM
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
-
+
-
+
-
- Application layer
- agent framework,
- end-user auth
+
+ Application layer
+ agent framework,
+ end-user auth
-
- ate-api-server
- OpenFGA authz, identity minting,
- objectstore-plugin sidecar
+
+ ate-api-server
+ OpenFGA authz, identity minting,
+ objectstore-plugin sidecar
-
-
- PostgreSQL
+
+
+ PostgreSQL
-
- ateworker + atunnel
- ingress listener, egress redirect,
- DNS relay
+
+ ateworker + atunnel
+ ingress listener, egress redirect,
+ DNS relay
-
- atelet DaemonSet
- objectstore-plugin sidecar
+
+ atelet DaemonSet
+ objectstore-plugin sidecar
-
- Golden actor
- built from the template
+
+ Golden actor
+ built from the template
-
-
- Snapshot bucket
+
+
+ Snapshot bucket
-
-
- Sandbox assets
- (SandboxConfig)
+
+
+ Sandbox assets
+ (SandboxConfig)
-
- Container image
- registry
+
+ Container image
+ registry
-
- P1
- create template
-
- P2
- run golden actor
-
- P3
- pull image
-
- P4
- start sandbox
-
- P5
- get actor certificate
-
- P6
- mint certificate
-
- P7
- suspend golden actor
-
- P8
- get storage credential
-
- P9
- upload snapshot
-
- P10
- copy tag snapshot
-
- P10
- mark ready
-
- P: prepare an actor template
-
- Other connection
-
- Sandbox: untrusted actor code
-
- Worker Pod: shared by many actors
-
- Node
-
- Control plane namespace
-
- Untrusted
-
- Node and worker components
-
- Control plane components
-
- Outside Substrate
+
+ P1
+ create template
+
+ P2
+ run golden actor
+
+ P3
+ pull image
+
+ P4
+ start sandbox
+
+ P5
+ get actor certificate
+
+ P6
+ mint certificate
+
+ P7
+ suspend golden actor
+
+ P8
+ get storage credential
+
+ P9
+ upload snapshot
+
+ P10
+ copy tag snapshot
+
+ P10
+ mark ready
+
+ P: prepare an actor template
+
+ Other connection
+
+ Sandbox: untrusted actor code
+
+ Worker Pod: shared by many actors
+
+ Node
+
+ Control plane namespace
+
+ Untrusted
+
+ Node and worker components
+
+ Control plane components
+
+ Outside Substrate
diff --git a/docs/assets/identity-actor-to-actor.svg b/docs/assets/identity-actor-to-actor.svg
new file mode 100644
index 0000000000..75ea853495
--- /dev/null
+++ b/docs/assets/identity-actor-to-actor.svg
@@ -0,0 +1,38 @@
+
diff --git a/docs/assets/identity-actor-to-cloud.svg b/docs/assets/identity-actor-to-cloud.svg
new file mode 100644
index 0000000000..d92fff226e
--- /dev/null
+++ b/docs/assets/identity-actor-to-cloud.svg
@@ -0,0 +1,36 @@
+
diff --git a/docs/assets/identity-actor-to-saas.svg b/docs/assets/identity-actor-to-saas.svg
new file mode 100644
index 0000000000..071f43bef3
--- /dev/null
+++ b/docs/assets/identity-actor-to-saas.svg
@@ -0,0 +1,34 @@
+
diff --git a/docs/assets/identity-sre-debugging.svg b/docs/assets/identity-sre-debugging.svg
new file mode 100644
index 0000000000..9488ec2904
--- /dev/null
+++ b/docs/assets/identity-sre-debugging.svg
@@ -0,0 +1,30 @@
+
diff --git a/docs/assets/identity-user-to-actor.svg b/docs/assets/identity-user-to-actor.svg
new file mode 100644
index 0000000000..3422ab746f
--- /dev/null
+++ b/docs/assets/identity-user-to-actor.svg
@@ -0,0 +1,35 @@
+
diff --git a/docs/assets/security-boundaries.svg b/docs/assets/security-boundaries.svg
index 6030a8010f..5121a3d2c2 100644
--- a/docs/assets/security-boundaries.svg
+++ b/docs/assets/security-boundaries.svg
@@ -2,84 +2,84 @@
Substrate security boundaries
-
-
-
-
-
+
+
+
+
+
-
+
-
- Substrate
-
- Control plane
-
- Worker node
-
- Sandbox
-
- Sandbox
+
+ Substrate
+
+ Control plane
+
+ Worker node
+
+ Sandbox
+
+ Sandbox
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
-
- Your application
- authenticates end users
+
+ Your application
+ authenticates end users
-
- Substrate API
- authorization per atespace
+
+ Substrate API
+ authorization per atespace
-
- Ingress gateway
- authenticated callers
+
+ Ingress gateway
+ authenticated callers
-
- Egress gateway
- destination policy,
- credential injection
+
+ Egress gateway
+ destination policy,
+ credential injection
-
- Actor A
- no credentials
+
+ Actor A
+ no credentials
-
- Actor B
- no credentials
+
+ Actor B
+ no credentials
-
- Node supervisor
+
+ Node supervisor
-
- External services
- and tools
+
+ External services
+ and tools
-
-
- Snapshot storage
+
+
+ Snapshot storage
- manage actors
- per-actor credentials (GCS)
-
- 1
-
- 1
-
- 2
-
- 2
-
- 3
-
- 4
- invoke
+ manage actors
+ per-actor credentials (GCS)
+
+ 1
+
+ 1
+
+ 2
+
+ 2
+
+ 3
+
+ 4
+ invoke
diff --git a/docs/guides/actor-jwt-federation.md b/docs/guides/actor-jwt-federation.md
index b56adca259..80c0292e36 100644
--- a/docs/guides/actor-jwt-federation.md
+++ b/docs/guides/actor-jwt-federation.md
@@ -67,6 +67,6 @@ If the relying party doesn't accept ES256,
To check the setup, have an egress policy inject a token for the relying
party's audience into a request to a server you control, and present that
-token to the relying party. Substrate can't yet present an actor JWT to a
-cloud token exchange endpoint itself; see
+token to the relying party. The gateway can't present an actor JWT to a cloud
+token exchange endpoint itself, but a custom credential provider can; see
[Known limitations](actor-jwts.md#known-limitations).
diff --git a/docs/guides/actor-jwts.md b/docs/guides/actor-jwts.md
index 300f03e48c..b8a9334b72 100644
--- a/docs/guides/actor-jwts.md
+++ b/docs/guides/actor-jwts.md
@@ -216,10 +216,12 @@ each step and what to do about a compromised key.
receives the token. An upstream that echoes request headers hands the token
back to the actor. A relying party should check `sub` or `actor.ate.dev` as
well as `aud`.
-- **No cloud token exchange.** Cloud token exchange endpoints take the JWT as
- a request parameter, and the gateway only replaces headers, so Substrate
- can't yet trade an actor JWT for cloud credentials
- ([#1661](https://github.com/agent-substrate/substrate/issues/1661)).
+- **No built-in cloud token exchange.** Cloud token exchange endpoints take
+ the JWT as a request parameter, and the gateway only replaces headers, so the
+ gateway can't trade an actor JWT for cloud credentials itself
+ ([#1661](https://github.com/agent-substrate/substrate/issues/1661)). A
+ custom credential provider can; see
+ [Actor → Cloud](../security/identity.md#actor--cloud).
- **A recreated actor can get a stale UID.** The gateway caches tokens by actor
name. For up to two thirds of a token's lifetime after an actor is recreated,
the gateway can inject a token whose `actorUID` names the old actor
diff --git a/docs/guides/authentication.md b/docs/guides/authentication.md
index c822a39ba0..f170813820 100644
--- a/docs/guides/authentication.md
+++ b/docs/guides/authentication.md
@@ -22,6 +22,9 @@ that principal may do.
`sub`.
- A call with no certificate and no valid token gets `Unauthenticated`.
+For identity in flows that don't call `ate-api-server`, such as actors calling
+other services, see the [Identity Overview](../security/identity.md).
+
## Which credential to use
| Caller | Credential | Principal |
diff --git a/docs/security/identity.md b/docs/security/identity.md
new file mode 100644
index 0000000000..08c16e6106
--- /dev/null
+++ b/docs/security/identity.md
@@ -0,0 +1,99 @@
+---
+title: "Identity Overview"
+linkTitle: "Identity Overview"
+weight: 4
+description: >
+ Identity system overview and explanation of common call flows.
+---
+
+# Identity overview
+
+Substrate identity is designed to easily integrate with external systems.
+IdP-issued identities for humans are recognized by Substrate, and
+Substrate-issued identities for actors are recognized by external APIs. In
+this document we explain the core identity flows. More advanced identity
+systems such as On-Behalf-Of
+([#3715](https://github.com/agent-substrate/substrate/issues/3715)) and ID-JAG
+([#3714](https://github.com/agent-substrate/substrate/issues/3714)) are being
+designed.
+
+| Flow | Credential | Issued by |
+|---|---|---|
+| [User → Actor](#user--actor) | The user's SSO token | Your human IdP |
+| [SRE / dev → Substrate](#sre--dev--substrate) | Short-lived OIDC token | The same human IdP |
+| [Actor → Actor](#actor--actor) | Actor JWT, injected at egress | Substrate |
+| [Actor → Cloud](#actor--cloud) | Actor JWT, federated via OIDC | Substrate, trusted by the cloud's Security Token Service (STS) |
+| [Actor → SaaS](#actor--saas-with-oauth) | OAuth access token, injected at egress | The SaaS OAuth server |
+
+## User → Actor
+
+
+
+1. Alice signs in to your app through your SSO. Your app frontend will make any
+ necessary connections to Substrate on her behalf.
+2. The frontend calls the ingress router (`atenet-router`) over mTLS. The
+ router only accepts the SPIFFE IDs listed in
+ [`ingressAllowedSPIFFEIDs`](../guides/operator-install.md#ingress-to-actors).
+3. The router routes the request to Actor A.
+
+Substrate does not see Alice's SSO token.
+
+## SRE / dev → Substrate
+
+
+
+1. The engineer gets a short-lived token from your SSO.
+2. The engineer calls `ate-api-server` with the token. The server trusts your
+ SSO issuer as a
+ [JWT provider](../guides/authentication.md#configuring-jwt-providers), and
+ [roles](../guides/authorization.md#roles) are granted per atespace.
+3. The engineer can inspect, suspend, and delete actors.
+
+## Actor → Actor
+
+
+
+1. Actor A calls Actor B through the egress gateway (`atenet-egress`).
+2. The gateway injects an [actor JWT](../guides/actor-jwts.md) for A and sends
+ the request to your app frontend.
+3. The frontend verifies the JWT and checks whether A may call B.
+4. The frontend calls the ingress router, which routes the request to Actor B.
+
+Actors can't reach each other directly. Your frontend authorizes every call.
+The router authorizes ingress per allowlisted client today, so an allowlisted
+frontend can reach every actor. Per-atespace ingress authorization will remove
+the need for the hop through the app frontend and is tracked in
+[#3589](https://github.com/agent-substrate/substrate/issues/3589).
+
+## Actor → Cloud
+
+
+
+1. Actor A sends a request with a placeholder header.
+2. The egress gateway asks a token exchange credential provider for a
+ credential.
+3. The provider exchanges A's actor JWT at the cloud STS, which
+ [trusts Substrate's issuer](../guides/actor-jwt-federation.md) through OIDC
+ federation.
+4. The gateway injects the cloud token.
+
+Substrate will support native token exchange soon
+([#1661](https://github.com/agent-substrate/substrate/issues/1661)). In the
+meantime, you can write a token exchange credential provider using the
+[credential provider API](../guides/egress-credential-injection.md#the-plugin-api).
+The provider gets the actor JWT by calling `MintActorJwt` with its own
+SPIFFE ID, which needs the
+[`egress_gateway` role](../guides/authorization.md#system-roles)
+(`--egress-gateway-spiffe-id`).
+
+## Actor → SaaS with OAuth
+
+
+
+1. Actor A sends a request with a placeholder header.
+2. The egress gateway fetches the access token from a
+ [credential provider](../guides/egress-credential-injection.md). The
+ provider gets it from an OAuth token broker, which holds each user's
+ refresh token. You can implement the broker yourself, or use one provided
+ by a cloud provider.
+3. The gateway injects the token.
diff --git a/docs/security/security.md b/docs/security/security.md
index 56232556bb..48f6e1439f 100644
--- a/docs/security/security.md
+++ b/docs/security/security.md
@@ -10,7 +10,7 @@ description: >
Substrate delivers zero trust for agents. Every agent has an identity, but no credentials, no open network, and no escape from the sandbox.
-Agent Substrate executes untrusted actors at scale in a multi-tenant environment where workloads are dynamic and memory states migrate continuously across physical infrastructure. All security controls operate within Substrate's cold-start and resume latency budget. This document defines the high-level security principles. For identity and credential configuration, see the [Authentication Guide](../guides/authentication.md). Detailed threat scenarios and mitigating invariants are documented [in the threat model](threat-model.md).
+Agent Substrate executes untrusted actors at scale in a multi-tenant environment where workloads are dynamic and memory states migrate continuously across physical infrastructure. All security controls operate within Substrate's cold-start and resume latency budget. This document defines the high-level security principles. For human and actor identities, see the [Identity Overview](identity.md). To configure access to `ate-api-server`, see the [Authentication Guide](../guides/authentication.md). Detailed threat scenarios and mitigating invariants are documented [in the threat model](threat-model.md).
## Security Boundaries
@@ -30,7 +30,7 @@ Your application is outside these boundaries: it authenticates end users and dec
Substrate provides the foundational base for agent security:
- Sandboxing: built-in compute, storage, and networking separation.
-- Identity: identity issuance is built-in and designed to federate with other identity systems. See the [Substrate Identity Critical User Journeys](https://docs.google.com/document/d/1seJNYQmsQYLCrYYzB9AmuxpSGSPCC3-u3tPNFsK2edc/edit?resourcekey=0-D6q8_0vqF6LPEk0BxWMTaA&tab=t.0#heading=h.6qjmtncwtihi).
+- Identity: identity issuance is built-in and designed to federate with other identity systems. See the [Identity Overview](identity.md).
- Gateway: built-in ingress and egress filtering that can do destination filtering and credential injection.
We expect the application layer creating actors using substrate to be responsible for: