Status: Accepted (v1) Date: 2026-01-21 Decision Makers: Engineering Related Docs:
WebHost.Systems/docs/spec/00_MASTER_SPEC.md
WebHost.Systems/docs/spec/10_API_CONTRACTS.md
WebHost.Systems/docs/spec/20_RUNTIME_PROVIDER_INTERFACE.md
WebHost.Systems/docs/spec/30_DATA_MODEL_CONVEX.md
WebHost.Systems/docs/spec/40_SECURITY_SECRETS_COMPLIANCE.md
WebHost.Systems/docs/spec/50_OBSERVABILITY_BILLING_LIMITS.md
webhost.systems is a multi-runtime AI agent platform. For each invocation executed in the data plane (Cloudflare Workers/DO or AWS Bedrock AgentCore), the platform must record usage and operational telemetry for:
plan enforcement (requests/tokens/compute limits),
cost estimation (v1: estimated cost),
billing integrity (tier gating and upgrade flows),
observability (error rates, latency/compute, tool usage),
audits and support (traceability via traceId, deploy version, runtime).
Telemetry is consumed by the control plane and stored as append-only raw events (metricsEvents) and aggregated views (billingUsage).
Telemetry is a critical security and business integrity surface. If telemetry is spoofable or not attributable, an attacker (or buggy runtime) could:
underreport usage to avoid limits or billing,
overreport usage to sabotage another tenant,
inject events for another tenant’s agents/deployments,
create ambiguity in chargeback attribution across runtimes,
break observability and incident response.
Therefore telemetry ingestion must ensure:
Integrity: payload not tampered in transit.
Authenticity: sender is an authorized runtime deployment.
Attribution: event belongs to exactly one {userId, agentId, deploymentId, runtimeProvider}.
Replay safety: retries do not double count; attackers cannot re-submit old events to alter usage.
No secrets leakage: telemetry must never include secret values, and telemetry auth secrets must not be stored as plaintext.
Constraints:
Data plane runtimes are heterogeneous and may differ in what usage details they can report.
Customer agent code is untrusted; it may attempt to forge telemetry or call arbitrary endpoints.
Telemetry ingestion must not rely on browser sessions/cookies.
The system must remain implementable for v1 without introducing a full centralized vault or complex PKI.
Adopt a deployment-scoped signed telemetry event model with ownership cross-checking at ingestion.
Attribution is mandatory
Every telemetry event MUST be attributable to:
userId
agentId
deploymentId
runtimeProvider
timestamp
Telemetry integrity uses deployment-scoped HMAC signatures (v1)
Each deployment has a unique telemetrySecret (HMAC key), generated by the control plane.
The telemetrySecret is injected into the deployed runtime using provider secret mechanisms.
Data plane submits telemetry to the control plane ingestion endpoint with:
header X-Telemetry-Deployment-Id: <deploymentId>
header X-Telemetry-Signature: v1=<hex(hmac_sha256(raw_body_bytes, telemetrySecret))>
Control plane verifies:
signature validity (using deployment-scoped secret reference),
ownership mapping: deployment belongs to the agent and user in the payload,
required fields and enums.
Ingestion performs ownership cross-check (defense in depth)
After signature validation, ingestion MUST query control plane data and verify:
deployments._id == deploymentId
deployments.agentId == agentId
deployments.userId == userId
If any mismatch: reject the event, optionally record an auditLog entry with sanitized details.
Telemetry does not use user auth
Telemetry ingestion MUST NOT accept browser session cookies as authentication. Telemetry is authenticated solely by deployment-scoped signature (and optional anti-replay).
Replay/dedupe (recommended in v1; mandatory before public invocations)
Telemetry ingestion SHOULD implement one of:
eventId (UUID generated by data plane) + short-window dedupe store, OR
dedupe on (deploymentId, traceId) if traceId uniqueness per invocation is guaranteed.
In either case, ingestion must be idempotent under retries to prevent double counting.
Secrets are never stored as plaintext in the primary DB
The primary DB stores only metadata and secret references (e.g., provider secret name/id).
The telemetrySecret value MUST NOT be stored in plaintext in the primary database (Supabase).
This ADR standardizes telemetry ingestion on deployment-scoped signed events for all runtimes. Provider capabilities vary, but the v1 baseline remains consistent.
Cloudflare (Workers + Durable Objects):
The runtime MUST explicitly emit telemetry to the control plane (Pattern A: in-workload emission is the preferred approach).
The telemetry signing secret is injected via Worker secret bindings.
The Worker/DO submits POST /v1/telemetry/report with X-Telemetry-Deployment-Id and X-Telemetry-Signature.
AgentCore (AWS Bedrock AgentCore):
AgentCore may provide provider-native observability capabilities (e.g., service-level telemetry/OTel-style integrations) that can be valuable for deep debugging.
v1 baseline: use the same direct signed submission model as Cloudflare (either in-workload emission when feasible, or adapter-side emission when required by provider constraints) to keep metering, limits, and billing semantics consistent across runtimes.
post-v1 enhancement: integrate provider-native observability for richer traces while still emitting the normalized, signed metering event used for billing and limits.
This approach satisfies the requirements with minimal complexity:
Integrity and authenticity: HMAC signatures prevent tampering and authenticate possession of a deployment-scoped secret.
Tenant protection: ownership cross-check prevents a compromised deployment secret from being used to write events for a different agent or user.
Least privilege: deployment-scoped secret limits blast radius if compromised.
Provider portability: works across runtimes because it relies on a generic “can send HTTP + store secret” capability.
Operable MVP: avoids the need for PKI or an always-online centralized vault, while still enabling strong guarantees.
Pros
Easiest to implement.
Cons
Enables trivial spoofing and billing manipulation.
Allows cross-tenant sabotage and incorrect usage enforcement.
Decision Rejected. Not acceptable for any paid system with limits and billing.
Telemetry is emitted only by the control plane adapter after it receives provider responses.
Pros
Avoids injecting telemetry secrets into runtime code.
Avoids runtime outbound call to control plane.
Cons
Often cannot capture provider-specific usage (compute/tool usage) reliably.
Creates attribution gaps if invocations happen outside the adapter path or if provider returns incomplete usage.
Weakens correlation with runtime events and can undercount.
Decision Not the primary model. Allowed as a constrained fallback if a provider cannot support in-workload emission, but must preserve integrity and attribution. If adapter emits, it must still attach to a deployment and enforce ownership; it must not be forgeable by clients.
Use one key per user shared across all deployments.
Pros
Fewer secrets.
Cons
Massive blast radius if compromised; compromised deployment can spoof across all user deployments.
Harder to rotate safely and isolate issues.
Decision Rejected. Deployment-scoped keys are safer and align with immutability and auditability.
Each deployment gets a keypair; telemetry signed with private key, verified with public key.
Pros
No shared secret at ingestion; avoids secret distribution to verifier.
Strong cryptographic model.
Cons
Adds key management complexity; still needs secure private key injection to runtime.
Higher implementation overhead for v1 without clear ROI.
Decision Deferred. Consider post-v1 if you need stronger non-repudiation or third-party verification.
Use mutual TLS with client certs per deployment.
Pros
Strong channel-level authentication.
Cons
Operationally heavy; certificate issuance/rotation complexity.
Hard to implement uniformly across runtimes.
Decision Deferred. Potential future hardening for enterprise tiers.
Telemetry payload MUST include:
userId (internal id)
agentId
deploymentId
runtimeProvider (cloudflare | agentcore)
timestamp (RFC3339 or unix ms; ingestion converts to unix ms)
requests (typically 1)
llmTokens (reported or estimated)
computeMs (wall time or billed time, defined per provider)
errors (0/1 typical)
errorClass (optional: auth | limit | runtime | tool | unknown)
costUsdEstimated
traceId (optional but recommended)
provider (optional runtime-specific counters)
Telemetry MUST NOT include:
plaintext secrets,
full prompts/responses by default (unless an explicit opt-in feature is added later),
raw provider credential material.
Signature is computed over the exact raw request body bytes as received/sent.
Algorithm: HMAC_SHA256(telemetrySecret, raw_body_bytes).
Encoding: hex lowercase recommended.
Header format: X-Telemetry-Signature: v1=<hex>.
Canonicalization note:
The verifier MUST use the raw body bytes, not a re-serialized JSON object.
If your framework modifies request bodies, capture raw bytes before parsing.
In order:
Read deploymentId header and locate deployment record.
Fetch deployment-scoped secret reference and verify signature.
Parse JSON body and validate required fields and enums.
Ownership cross-check:
deployments.userId == body.userId
deployments.agentId == body.agentId
deployments._id == body.deploymentId (or match header)
Dedupe/idempotency:
if eventId used: reject or no-op if already processed in dedupe window
if traceId used: ensure uniqueness per (deploymentId, traceId) in dedupe window
Persist event append-only.
Optionally update incremental counters.
If any step fails:
respond with normalized error (for non-telemetry callers, keep generic),
DO NOT reveal secret or internal details,
optionally write a sanitized auditLog entry for investigation.
Generate a new telemetry secret on every new deployment by default.
Inject the secret into the runtime provider as a secret (not as plain env var stored in DB).
Rotation policy:
New deployment => new telemetry secret (mandatory).
Old deployments retain old secret for the duration they might emit telemetry (but they should not emit after deactivation, except in-flight retries).
Keep verification capability for a bounded time window after deactivation to allow late telemetry retries.
Data plane SHOULD retry telemetry submission with exponential backoff (bounded).
Data plane SHOULD NOT block user-visible invocation completion while retrying telemetry.
Control plane SHOULD expose internal health metrics for telemetry ingestion rate and rejection counts.
Strong billing and limits integrity foundation.
Provider-agnostic design that scales to new runtimes.
Reduced blast radius via deployment-scoped secrets.
Clear audit path for telemetry failures and spoof attempts.
Requires deploying a telemetry secret per deployment and managing injection per provider.
Requires careful handling of raw request bytes for correct signature verification.
Adds complexity to runtime code (Pattern A emission) or to adapters (Pattern B emission).
[ ] Each deployment has a telemetry signing secret generated by the control plane.
[ ] Secret is injected into the runtime provider using provider secret mechanisms.
[ ] Telemetry endpoint verifies signature using raw request body bytes.
[ ] Telemetry endpoint cross-checks {userId, agentId, deploymentId} ownership.
[ ] Telemetry endpoint is not authenticated via browser sessions/cookies.
[ ] Telemetry ingestion is idempotent under retries (dedupe window).
[ ] Telemetry payloads do not include secret values or full prompt contents by default.
[ ] Telemetry rejection and signature failures are auditable (sanitized audit log).
This ADR is considered correctly implemented when:
A valid telemetry payload signed with the correct deployment secret is accepted and persisted.
The same payload with an invalid signature is rejected and not persisted.
A valid signature with mismatched ownership (deployment belongs to a different agent/user) is rejected.
Retried telemetry submissions do not cause double-counting within the dedupe strategy window.
No plaintext telemetry secret values exist in the primary DB.
Telemetry events are attributable to {userId, agentId, deploymentId, runtimeProvider} and can be aggregated reliably for billing and limits.
ADR-0005: Deployment immutability + active pointer model (if not already documented as ADR).
Define a standardized eventId policy if traceId uniqueness cannot be guaranteed.
Consider stronger anti-replay guarantees (timestamp window + dedupe store) before enabling public agent invocations.
Consider future hardening for enterprise tiers (PKI signatures or mTLS) if required by compliance/security posture.