Status: Accepted
Date: 2026-01-21
Owners: Engineering
Related docs:
WebHost.Systems/docs/spec/00_MASTER_SPEC.md
WebHost.Systems/docs/spec/10_API_CONTRACTS.md
WebHost.Systems/docs/spec/20_RUNTIME_PROVIDER_INTERFACE.md
WebHost.Systems/docs/spec/40_SECURITY_SECRETS_COMPLIANCE.md
webhost.systems is a multi-runtime AI agent deployment platform with:
Control plane: Auth, dashboard, agent/deployment metadata, billing, limits, metrics aggregation.
Data plane: Executes customer agent code on runtime providers (initially Cloudflare Workers/DO and AWS Bedrock AgentCore).
The platform must handle several categories of secrets:
Customer secrets: model keys (e.g., OPENAI_API_KEY), third-party API keys used by agent code.
Platform secrets: Cloudflare API tokens, AWS credentials/roles, billing webhook secret, internal signing keys.
Integrity secrets: telemetry signing keys (deployment-scoped).
Constraints:
Customer agent code is untrusted and may exfiltrate secrets it is given; therefore secrets must be scoped and minimized.
The platform must support multiple runtimes, each with different secret injection mechanisms.
The platform must enable telemetry integrity: data plane must report usage to control plane without allowing spoofing.
Security and operational requirements:
Minimize blast radius of a control-plane DB read breach.
Prevent secret leakage via logs, errors, metrics, or UI.
Support secret rotation with minimal operational burden.
Keep implementation feasible for MVP while providing a clear path to stronger controls.
No plaintext secrets are stored in the primary application database (Supabase).
The DB stores only:
secret key names (e.g., envVarKeys),
provider secret references (names/ids),
rotation metadata (timestamps, versions),
booleans like “is configured”.
Secrets are injected into runtime providers using provider-native secret mechanisms (where possible).
Cloudflare: Worker secret bindings (set via Cloudflare API).
AWS / AgentCore: AgentCore Runtime environment variable injection via the AgentCore Runtime API (recommended default for v1), with AWS Secrets Manager references as an optional enhancement for advanced rotation and centralized secret management.
Telemetry integrity uses deployment-scoped signing keys injected into the runtime as secrets.
Each deployment gets a unique telemetrySecret (HMAC key).
Data plane signs telemetry payloads with HMAC-SHA256 over raw request bytes.
Control plane verifies signature using the secret reference associated with the deployment.
Control plane never exposes secret values back to clients.
A “set secrets” endpoint may exist and is write-only.
A “get secrets” endpoint for plaintext MUST NOT exist.
The UI can show only:
configured keys list,
status per key (set/not set),
last rotated timestamp (optional).
Reduces breach impact: If the primary DB is compromised, attackers do not automatically obtain usable API keys.
Aligns with best practices: Provider secret stores are purpose-built for secret storage and rotation.
Enables multi-runtime: Provider injection allows per-runtime deployment without inventing a bespoke secret distribution system.
Supports telemetry integrity: Deployment-scoped secrets and HMAC signatures prevent spoofing of usage events, which protects billing and limit enforcement.
Keeps MVP achievable: Write-only secrets entry + provider injection is implementable without building a full secrets vault and policy engine.
Pros
Simple read/write path.
Centralized regardless of runtime provider.
Cons
Key management becomes a major responsibility (KMS integration, rotation, access controls).
If application and DB are compromised, secrets may still be exposed via runtime memory or logs.
More complex than necessary for MVP given provider secret facilities.
Decision: Rejected for v1 (may revisit if a centralized secrets vault is required).
Pros
Single system of record, policy controls, potential audit/compliance benefits.
Cons
Adds significant infrastructure and operational complexity.
Requires secure runtime access and network paths from every provider.
Introduces availability coupling: vault outage can break invocations.
Decision: Rejected for v1; consider for enterprise roadmap if needed.
Pros
Easiest to implement.
Cons
Catastrophic: secrets leak via artifacts, logs, and provider introspection.
Violates core security requirement.
Decision: Rejected.
Pros
Avoids injecting telemetry secret into runtime.
Cons
Often cannot capture accurate compute/tool usage.
Harder to attribute reliably at high concurrency; may undercount or miscount.
Still requires strong integrity controls in adapter pathway.
Decision: Not the primary strategy. Allowed only as a constrained fallback if provider limitations require it, but must preserve attribution integrity.
Stronger baseline security posture without heavy infrastructure.
Clear separation of responsibilities:
DB stores metadata and references,
providers store values.
Enables per-deployment telemetry signing and robust anti-spoofing.
Secret management becomes provider-dependent; implementation must support multiple injection mechanisms.
Secret rotation may require redeploy on some providers.
Debugging becomes harder because plaintext is not retrievable; must rely on “configured/not configured” and test invocations.
agents.envVarKeys: list of secret key names (never values).
deployments.telemetryAuthRef: provider reference to telemetry signing secret (never plaintext).
Optional secrets metadata table (if implemented) stores only:
userId, agentId, key, provider, secretRef, createdAtMs, rotatedAtMs.
If implementing POST /v1/agents/{agentId}/secrets (write-only):
MUST authenticate and enforce tenant ownership.
MUST validate keys against agents.envVarKeys (or allow adding new keys via agents.update first).
MUST store values only in provider secret store(s) and discard plaintext immediately after injection.
MUST NOT return secret values.
MUST redact secret values in logs and error messages.
Cloudflare adapter:
Use provider API to set Worker secrets per deployment or per agent, and ensure the active Worker uses the correct secrets.
AgentCore adapter:
Use AgentCore Runtime environment variable injection (v1 default) to set deployment-scoped secrets (telemetry signing key, user-provided API keys) as runtime environment variables.
Use AWS Secrets Manager references as an optional enhancement (post-v1 or enterprise hardening) to support rotation without redeploy and centralized secret governance.
Store only references/metadata in the control plane DB (e.g., runtime env var key names and/or Secrets Manager ARNs). Never store plaintext values.
AgentCore supports multiple secret injection patterns. For v1, prefer environment variables because it minimizes moving parts and matches the Cloudflare deployment model (provider-managed secret injection at deploy time).
Option 1 (v1 default): Environment variables
Control plane sets runtime environment variables as part of the deploy/create/update runtime call.
Example variables:
TELEMETRY_SECRET (deployment-scoped HMAC key)
user-provided keys like OPENAI_API_KEY
Operational note: rotation may require redeploy depending on provider behavior; document in UI.
Option 2 (optional): AWS Secrets Manager references
Store secret values in Secrets Manager.
Provide the runtime with references (ARNs) and an IAM role that permits retrieval.
Operational note: enables rotation without redeploy, but increases IAM + operational complexity.
Each deployment MUST have a unique telemetry signing secret.
Runtime sends telemetry with:
X-Telemetry-Deployment-Id: <deploymentId>
X-Telemetry-Signature: v1=<hex-hmac-sha256(body)>
Control plane MUST verify:
signature validity,
deployment ownership mapping to {userId, agentId},
required attribution fields.
Telemetry secrets:
MUST rotate on every new deployment by default.
Customer secrets:
MAY be rotated by user action; system SHOULD record rotation timestamp.
If provider requires redeploy for new secret values to take effect, UI MUST make this explicit.
This ADR supports a SOC2-aligned posture by:
minimizing sensitive data stored in primary DB,
ensuring integrity of billing-relevant telemetry,
enabling auditability (via audit logs for secrets set and telemetry rejections).
This ADR does not eliminate the inherent risk that untrusted customer agent code can exfiltrate secrets it is given; the platform mitigates by scoping secrets per agent and avoiding cross-tenant exposure.
This ADR is correctly implemented when:
No plaintext secret values exist in the primary DB (verified via code review and automated checks).
Secrets set via API are write-only; plaintext secrets are never returned.
Provider secrets are injected and required for runtime invocation (e.g., model keys, telemetry signing secret).
Telemetry ingestion rejects invalid signatures and mismatched ownership.
Logs and error messages are sanitized and do not contain secret values.
Secret rotation is supported at least via “set new value” flow and telemetry secrets rotate per deployment.