Canonical public product/domain: e.computedriven.com Initial analytical freshness target: 2 hours Primary data plane: Cloudflare Workers + Cloudflare Queues + ClickHouse Cloud Future live plane: Durable Objects + WebSockets + Super Date: 2026-08-23 Status: architecture/review brief; Opus must inspect the current codebase before implementing
The public product surface is:
`e.computedriven.com`
It should contain:
e.computedriven.com
├── marketing
├── docs
├── app/dashboard
├── public API
├── SDK documentation
└── canonical fallback ingest endpointFor ComputeDriven-owned sites, prefer a first-party browser ingestion route on each site:
https://computedriven.com/_e/v1/events
https://opensentience.org/_e/v1/events
https://ampersandboxdesign.com/_e/v1/events
...Those routes should terminate in Cloudflare and forward through a Worker/service binding to the same central E ingest logic.
External or non-Cloudflare integrations can later use:
https://e.computedriven.com/v1/eventsor configure their own first-party proxy.
E is one coherent product surface. The ingest endpoint, developer documentation, dashboard, and eventual live-context experience should all be designed as parts of that product rather than as separate products.
E is not “our own Google Analytics.”
It is the event, journey, and live-context plane for ComputeDriven.
The long-term idea is:
Behavior should not end in a dashboard. It should become governed context that humans and agents can understand and act on while the user is still present.
The short-term system is deliberately simpler:
browser / server events
|
v
Cloudflare Workers
|
v
Cloudflare Queue
|
| buffered
| every 2 hours
v
ClickHouse Cloud
|
+----> E dashboard
+----> PostHog
+----> Amplitude
+----> cached summaries
|
`----> later Super projectionsThe critical decisions are:
E owns the canonical semantic event contract.
E owns stable event IDs.
ClickHouse is an isolated analytics database from day one.
Existing ComputeDriven product databases are not part of this scaling/failure domain.
PostHog and Amplitude are downstream analytical lenses, not sources of truth.
Live session state will later live outside ClickHouse.
Historical analytical freshness starts at two hours, not seconds.
We deliberately learn Workers + Queues + ClickHouse now while mistakes are cheap.
Do not put E event tables in the existing ComputeDriven Pigsty deployment.
Do not use the Keycloak/PlanetScale database.
Do not create hidden cross-database coupling.
Conceptually:
COMPUTEDRIVEN PRODUCT PLANE
---------------------------
identity
accounts
organizations
worlds
runs
nodes
billing
product state
existing Pigsty / product stores
|
| emits approved semantic events
v
E ANALYTICS PLANE
-----------------
Cloudflare Workers
Cloudflare Queues
ClickHouse Cloud
later Durable Objects
later R2 archiveThe product plane may publish events such as:
signup_completed
workspace_created
world_created
build_completed
subscription_started
subscription_cancelledbut E must not require direct table reads from the product database to function.
ClickHouse should hold:
accepted semantic events;
analytical session/journey facts;
attribution facts;
bot-intervention analytical events;
derived/materialized analytical views.
It must not become authoritative for:
auth;
organizations;
billing entitlement;
World authority;
secrets;
production mutations.
Do not keep ClickHouse awake merely to answer:
who is on this page now?
what did this visitor click 3 seconds ago?
is chat open?
which component is visible?Those are later Worker/Durable Object concerns.
ClickHouse initially answers:
what happened in the last 2h/day/week/month?
which journeys convert?
where do funnels fail?
which sources lead to signup?
which events predict activation?Before changing code, inspect the current repository/repositories and write a short findings note.
Opus must determine:
Where the current ComputeDriven marketing site lives.
Whether e.computedriven.com already has DNS/routes/code.
Current Cloudflare Worker/Pages conventions.
Current package manager / monorepo layout.
Current TypeScript/shared-schema conventions.
Existing event/analytics abstractions, if any.
Existing PostHog, Amplitude, GA, Cloudflare Web Analytics, or custom analytics code.
Existing consent/privacy implementation.
Existing Cloudflare account resources managed by Wrangler/Terraform/Pulumi/manual configuration.
Existing CI/deploy patterns.
Current secrets-management convention.
Current ComputeDriven identity IDs that can safely become opaque analytics IDs.
Any existing cross-domain linking/UTM utilities.
Whether current sites are all on Cloudflare and can expose first-party /_e/* routes.
Any conflict between this brief and newer Super/ComputeDriven architecture already present in code.
Do not duplicate existing abstractions.
If the repo materially contradicts this brief, report the contradiction before “fixing” it.
Recommended v1:
ComputeDriven-owned sites
┌─────────────────────────────────────┐
│ computedriven.com │
│ opensentience.org │
│ ampersandboxdesign.com │
│ ... │
└─────────────────┬───────────────────┘
│
/_e/v1/events
│
v
Cloudflare route/Worker
│
service binding / shared lib
│
v
┌──────────────────────────┐
│ cd-e-ingest Worker │
│ │
│ validate │
│ normalize │
│ redact │
│ consent │
│ rate-limit │
│ batch checks │
└────────────┬─────────────┘
│
v
┌──────────────────┐
│ e-events Queue │
│ │
│ 2h buffer target │
└────────┬─────────┘
│
scheduled pull every 2h
│
v
┌────────────────────┐
│ cd-e-flush Worker │
│ Cron Trigger │
│ HTTP pull consumer │
└─────────┬──────────┘
│
v
┌──────────────────┐
│ ClickHouse Cloud │
│ Basic initially │
└───────┬──────────┘
│
┌──────────────┼────────────────┐
│ │ │
v v v
materialized fanout state snapshots/cache
views │
├── PostHog
└── AmplitudePublic product:
e.computedriven.com
|
+-- /
+-- /live
+-- /architecture
+-- /docs
+-- /privacy
+-- /app
+-- /v1/events fallback public ingestDo not attach the canonical event queue to a normal push consumer if the objective is to let ClickHouse sleep.
A push consumer automatically consumes continuously. Delaying each message by two hours only creates a two-hour-offset continuous stream; after the initial delay it can still keep ClickHouse awake around the clock.
The intended V1 behavior is:
00:00 - 01:59 Queue accumulates
02:00 flush wakes ClickHouse
02:00-02:0x drain queue + insert
02:0x snapshot/materialized work complete
02:0x+idle ClickHouse scales down
02:00 - 03:59 Queue accumulates
04:00 next flush
...Use:
Cloudflare Queue configured with HTTP pull consumer;
Cloudflare Cron Trigger on a dedicated flush Worker;
cron expression conceptually every two hours;
scheduled Worker repeatedly pulls batches until:
queue is drained;
execution time budget is nearly exhausted;
ClickHouse failure occurs;
a safety/backpressure threshold is reached.
Cloudflare documents pull consumers specifically for cases where upstream consumption needs to be rate/availability controlled. Pull batch size currently supports up to 100 messages. Queue retention on Paid can be configured up to 14 days, giving ample recovery room for a 2-hour cadence.
Cloudflare Cron Triggers run in UTC.
Do not encode business meaning into the exact UTC boundary. “Every two hours” is an operational flush interval, not a reporting-day definition.
V1 historical analytical SLO:
normal freshness target: <= 2h 15mReason:
up to ~2h wait before scheduled drain;
ClickHouse cold start;
drain/insert/materialization time;
cached dashboard snapshot publication.
Track:
oldest_unflushed_event_age_seconds
queue_depth
last_successful_flush_at
last_successful_event_occurred_at
clickhouse_flush_duration_ms
events_flushed
batches_flushedAlert thresholds:
WARN oldest event > 2h 30m
ERROR oldest event > 4h
CRITICAL oldest event > 8hThe queue retention must be much longer than the critical threshold.
Do not equate CDN HTTP requests with analytics events.
Do not send one Worker request for every semantic event unless a specific event requires immediate delivery.
Preferred SDK behavior:
browser session
|
+-- event
+-- event
+-- event
+-- event
|
v
microbatch
|
+-- timer
+-- pagehide
+-- size threshold
v
/_e/v1/eventsInitial targets:
max events per browser batch: 25
target payload: < 64 KB
hard payload cap: significantly below Worker/Queue limits
flush timer: 15-30 seconds
flush on pagehide: yes
sendBeacon/keepalive fallback: where appropriateWhy target under 64 KB:
Cloudflare Queue operations are metered in 64 KB chunks. Keeping a logical browser batch below that boundary makes cost and behavior simpler.
Do not obsess over exactly 25. Measure actual encoded event sizes and tune.
E owns the schema.
Suggested v1:
{
"schema_version": 1,
"event_id": "01J...",
"event_name": "pricing_viewed",
"occurred_at": "2026-08-23T16:30:00.000Z",
"site_id": "computedriven",
"domain": "computedriven.com",
"path": "/pricing",
"anonymous_id": "anon_...",
"user_id": null,
"account_id": null,
"session_id": "ses_...",
"journey_id": null,
"source": {
"referrer": "https://opensentience.org/...",
"source_domain": "opensentience.org",
"utm_source": null,
"utm_medium": null,
"utm_campaign": null
},
"context": {
"locale": "en-US",
"viewport_class": "desktop",
"release": "git-sha"
},
"consent": {
"analytics": true,
"replay": false,
"personalization": false
},
"properties": {
"placement": "hero"
}
}Rules:
event_id is immutable.
Event names are stable snake_case.
Reject unsupported schema versions.
Never use email as the canonical user key.
user_id/account_id are opaque internal IDs.
Keep top-level fields tightly controlled.
Free-form properties have max nesting/bytes.
URLs/query strings are redacted/allowlisted.
Server strips known sensitive keys even if client validation fails.
Historical event semantics are never silently redefined.
V1 canonical events should remain intentionally small.
Suggested core:
page_viewed
cta_clicked
docs_opened
research_opened
pricing_viewed
signup_started
signup_completed
login_completed
workspace_created
project_created
world_created
first_build_started
first_build_completed
checkout_started
subscription_started
subscription_upgraded
subscription_cancelled
chat_started
chat_question_asked
bot_help_offered
bot_help_accepted
bot_action_proposed
bot_action_completed
bot_action_rejectedDo not put:
mousemove
every scroll tick
every DOM mutation
raw replay frameinto the canonical ClickHouse semantic lane.
High-resolution behavioral exhaust remains primarily a PostHog concern.
Use ClickHouse Cloud Basic initially unless repo/security requirements force Scale.
Current public AWS us-east-1 list pricing checked 2026-08-23:
Basic compute: $0.2181 / compute unit-hour
Basic storage: $25.30 / TB-month
Basic storage: up to 1 TB
Basic memory: 8-12 GiB total
Basic AZs: 1ClickHouse states that unused resources can scale down to zero.
This is intentionally a learning/starter deployment. Cloudflare Queues isolates event acceptance from ClickHouse availability.
Upgrade triggers to Scale should be measurement-driven:
need >1 TB
need stronger production availability
need 2+ AZ
need configurable memory / compute separation
need private networking
need workload isolationDo not self-host ClickHouse yet merely to save tens of dollars.
The point of V1 is to learn:
schema design;
sort keys;
partitions;
inserts;
async inserts;
idempotency;
materialized views;
query behavior;
compression;
cost per million events;
cold starts;
scale-to-zero behavior;
operational failure modes.
For Worker access, use the official web/fetch client unless repo testing shows a reason not to:
@clickhouse/client-webThe official ClickHouse JavaScript client documents Cloudflare Workers compatibility.
Keep all ClickHouse access server-side.
Never ship ClickHouse credentials to browser code.
Create distinct ClickHouse users/roles:
e_ingest
INSERT approved raw tables
minimal SELECT if required for health
no broad ALTER
e_query
SELECT approved tables/views
no INSERT
no ALTER
e_admin
migrations/schema
tightly held
e_super later
SELECT only explicitly approved viewsDo not give Super raw arbitrary SQL authority.
Opus should benchmark actual query patterns before freezing ORDER BY.
A candidate—not a final ruling:
CREATE TABLE events_v1
(
event_id UUID,
batch_id UUID,
schema_version UInt16,
event_name LowCardinality(String),
occurred_at DateTime64(3, 'UTC'),
received_at DateTime64(3, 'UTC'),
site_id LowCardinality(String),
domain LowCardinality(String),
path String,
anonymous_id String,
user_id Nullable(String),
account_id Nullable(String),
session_id String,
journey_id Nullable(String),
source_domain LowCardinality(String),
referrer String,
context JSON,
consent JSON,
properties JSON
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(occurred_at)
ORDER BY
(
site_id,
toDate(occurred_at),
event_name,
occurred_at,
session_id,
event_id
);Benchmark alternatives based on real intended queries, especially:
events for one session
events for one user
events by site and time
events by event type and time
funnel sequences
source -> conversion
journey traversal
bot-assisted conversionWrite a short benchmark note before finalizing the key.
The Queue should carry microbatches, not single events.
Example:
{
"batch_version": 1,
"batch_id": "01J...",
"site_id": "computedriven",
"accepted_at": "2026-08-23T16:30:02.000Z",
"events": [
{ "...": "..." },
{ "...": "..." }
]
}Requirements:
one batch_id per accepted Worker microbatch;
every event also has its own event_id;
target message size <64 KB;
hard max event count;
hard max bytes;
invalid event handling is explicit;
never queue unchecked arbitrary payloads.
Pseudo-flow:
scheduled() every 2h
|
v
acquire flush lease / avoid overlapping drain
|
v
wake/query ClickHouse
|
v
repeat:
pull <=100 queue messages
if empty -> stop
flatten validated event batches
bound aggregate insert bytes/rows
INSERT into ClickHouse
if INSERT confirmed:
ack pulled queue messages
else:
do not ack
exit / allow visibility timeout
until time budget reached
|
v
refresh materialized/snapshot outputs if needed
|
v
write cheap dashboard freshness snapshot
|
v
end; ClickHouse becomes idleCorrect:
pull -> insert -> confirmed -> ackWrong:
pull -> ack -> insertThe wrong order can lose analytics on a failed insert.
Assume at-least-once delivery.
Design ClickHouse insert/idempotency handling so a Worker crash after database success but before queue ACK does not silently duplicate analytical meaning.
Test this explicitly.
Do not claim “exactly once” unless the entire path actually proves it.
The database only stays cheap if E does not constantly wake it.
Therefore:
WRITE PATH
queue -> ClickHouse every ~2h
NORMAL PUBLIC READ PATH
cached snapshots / cheap edge state
DEEP ANALYTICS READ PATH
explicit ClickHouse queryAt the end of a successful flush, publish summary data required for the ordinary dashboard to a cheap always-available layer.
Possible v1 choices after repo inspection:
Cloudflare KV
D1
Cache API
static JSON snapshot in R2 laterDo not build a second analytical database there.
Store only dashboard-serving summaries such as:
last_flush_at
events_last_24h
pageviews_today
signups_today
top_sites
top_pages
conversion snapshot
vendor delivery healthOpening the public E homepage should not wake ClickHouse.
Do not promise these numbers. Measure them.
At the current portfolio traffic level, CDN requests are not equal to analytics events.
If semantic browser/server events are batched before entering Workers, Cloudflare should remain inexpensive.
Cloudflare Workers Paid currently lists:
$5/month base
10M Worker requests/month included
$0.30 per additional million requests
30M CPU-ms/month includedCloudflare Queues Paid currently lists:
1M operations/month included
$0.40 per additional million operations
typical successful message ~= write + read + deleteClickHouse Cloud Basic currently lists:
$0.2181 / compute unit-hour in AWS us-east-1
$25.30 / TB-month storageThe exact cost of a two-hour wake pattern depends on:
service size;
resume latency;
insert duration;
merge/background activity;
idle timeout;
dashboard queries;
batch size;
actual analytics event volume.
Initial engineering target:
keep the core E data plane in roughly the $10-$30/month range if real measurements support it, without sacrificing correctness merely to hit that number.
The software must expose actual:
ClickHouse active minutes/day
events per flush
bytes per flush
compressed bytes/event
Worker requests/month
Queue operations/month
cost per 1M accepted eventsso future cost decisions are empirical.
The canonical chain should be:
browser
|
v
E ingress
|
v
Queue
|
v
ClickHouse accepted event
|
+---> PostHog
`---> AmplitudeE is the semantic source of truth.
PostHog receives:
canonical semantic events;
plus its own high-resolution browser behavior/replay path where configured.
Amplitude receives:
curated canonical semantic events;
not raw behavioral exhaust.
Use stable E event_id downstream for dedup/reference.
Do not scatter direct semantic posthog.capture() and Amplitude calls throughout site code if they bypass E’s event contract.
Vendor-specific behavioral features may remain vendor-specific.
Both PostHog and Amplitude must be able to see anonymous -> authenticated journeys.
Do not split:
guests -> PostHog
members -> AmplitudeUse:
before login:
anonymous_id = anon_x
user_id = null
after login:
anonymous_id = anon_x
user_id = usr_yRecord an identity link according to privacy policy.
On logout:
reset vendor identity state;
prevent a later browser user from inheriting the prior authenticated identity.
Never use email address as the analytics primary key.
Cookies do not become first-party across unrelated registrable domains.
Do not use fingerprinting or hidden third-party-cookie hacks.
For known portfolio links:
source domain
|
signed short-lived journey token
|
v
destination domain
|
local anonymous session linked to journey_idThe token:
contains no secrets;
contains no email;
expires quickly;
is signed;
is removed from the visible URL after exchange where practical;
cannot magically deanonymize unrelated traffic.
Example conceptual payload:
{
"v": 1,
"journey_id": "jny_...",
"source_site": "opensentience",
"source_session": "ses_...",
"iat": 1787500000,
"exp": 1787500600
}/_e routingFor ComputeDriven-owned Cloudflare zones, investigate a reusable deployment pattern:
domain.com/_e/*
|
v
zone Worker route
|
v
service binding to cd-e-ingestBenefits:
first-party browser endpoint;
no cross-origin ceremony for our own sites;
less obvious third-party tracking hostname;
one central validation implementation;
site identity can be bound by route/config instead of trusting arbitrary browser input.
Security rule:
The ingest Worker should derive/verify
site_idfrom trusted route/config where possible, not blindly trust client-suppliedsite_id.
Fallback:
e.computedriven.com/v1/eventsfor integrations where first-party routing is unavailable.
e.computedriven.com websiteE is both product surface and architecture demonstration.
Suggested routes:
/
/live
/architecture
/docs
/privacy
/app
/app/events
/app/funnels
/app/journeys
/app/sources
/app/system
/v1/events
/healthInitial public positioning:
Eyebrow
LIVE BEHAVIORAL CONTEXT
Hero
Your analytics should be in the room.
Subhead
Instrument once. Understand the journey. Turn behavior into governed context that humans and agents can use.
Alternative:
Understand the journey. Join it.
Do not market V1 features as live if the ClickHouse analytical layer is intentionally two hours fresh.
Explain the two planes honestly:
historical analytics: buffered / ~2h
live context: future/current separate live planeShow the E event flow.
One E event -> ClickHouse -> PostHog -> Amplitude.
Show:
live session state;
analytical history.
Show portfolio link flow.
Vendor independence.
Preview future Super visibility/capability rules.
Interactive technical diagram.
SDK/API snippets only when actual implementation exists.
Specific redaction/consent model.
Turn behavior into context.
Do not use fake customer logos, fake counts, or fake “real-time” claims.
This is not required for initial ClickHouse ingestion, but preserve the seam.
Later:
browser
|
v
Session Durable Object
|
+-- WebSocket
+-- current page
+-- recent semantic events
+-- chat
+-- current SessionProjection
+-- typed website capabilities
|
v
SuperClickHouse remains historical evidence/analytics.
Durable Object remains live session state.
Do not force ClickHouse to become a low-latency actor/session store.
The eventual assistant receives a scoped projection, not browser omniscience.
Example:
{
"identity": {
"kind": "anonymous"
},
"page": {
"path": "/pricing",
"visible_components": ["plans", "local-vs-cloud"]
},
"journey": {
"recent_events": [
"architecture_opened",
"pricing_viewed"
]
},
"capabilities": [
"navigate",
"highlight",
"open_docs",
"compare_plans",
"start_signup"
]
}Never include by default:
password contents
payment card fields
API keys
secrets
clipboard
arbitrary unsent form contents
unrelated organization data
raw database accessBots should operate typed UI capabilities, not arbitrary JavaScript.
Define explicit categories:
analytics
replay
personalization
support_chatDo not hardcode legal conclusions.
Engineering requirements:
browser consent API;
vendor initialization gated by policy;
replay separately governed;
server-side redaction even if client is buggy;
query-string allowlist/redaction;
deletion path by opaque identity;
retention policy configuration;
internal/bot traffic filtering;
privacy documentation based on implementation.
Minimum requirements:
restrictive allowed origins for fallback cross-origin ingest;
first-party Worker routes where possible;
request body limits;
event count limits;
property nesting/size limits;
schema validation;
sensitive-key redaction;
rate limits;
bot/abuse handling;
Worker secrets for ClickHouse credentials;
separate ClickHouse users/roles;
TLS only;
no browser DB credentials;
no public admin SQL endpoint;
CSP on E website;
authenticated protected /app;
CSRF where browser state changes exist;
no sensitive event payloads in ordinary logs;
environment separation:
dev;
staging;
prod;
separate queues and ClickHouse DB/table prefixes per environment.
Preserve existing monorepo conventions if present.
Conceptually:
apps/
e-web/
e-ingest-worker/
e-flush-worker/
e-api-worker/ optional if separate
packages/
e-schema/
e-sdk/
e-clickhouse/
e-privacy/
e-test-fixtures/
infra/
cloudflare/
clickhouse/If existing repo style differs, adapt rather than force this shape.
Important separation:
e-schemamust be shared by:
browser SDK;
ingest validation;
tests;
ClickHouse mapping;
vendor adapters.
Do not ship the happy path only.
Invalid schema never reaches Queue.
Unsupported schema version is rejected.
Oversized batch is rejected.
Oversized property object is rejected.
Sensitive configured keys are stripped server-side.
site_id cannot be spoofed on first-party routes.
Duplicate event_id retry does not create double analytical meaning.
Page navigation does not block waiting on analytics.
Queue can accumulate for >2h without loss.
Flush Worker does not run continuously.
No push consumer exists on canonical buffered queue.
Cron overlapping with previous drain is safely serialized or made harmless.
Empty queue does not keep ClickHouse awake with useless work.
Backlog survives a ClickHouse outage longer than one interval.
Messages are not ACKed before successful DB insert.
Worker crash after DB insert but before ACK is safe/idempotent.
Visibility timeout causes safe redelivery.
Backlog age alarms fire.
Candidate sort keys are benchmarked against representative queries.
Materialized view failure behavior is understood/tested.
Cold-start duration is measured.
Idle scale-down is observed rather than assumed.
Insert batch size is measured for throughput/cost.
Raw event query remains correct across month partitions.
Query user cannot INSERT.
Ingest user cannot perform broad admin operations.
Anonymous -> authenticated link works.
Logout prevents identity bleed to the next user.
Two users on one browser do not collapse.
Cross-domain journey token expires.
Tampered journey token fails.
Unrelated anonymous visits are not automatically linked.
ClickHouse succeeds while PostHog fails: canonical event remains valid.
ClickHouse succeeds while Amplitude fails: canonical event remains valid.
Vendor retry reuses E event ID.
One vendor outage does not block the other.
E homepage does not wake ClickHouse under normal cached rendering.
Dashboard indicates freshness timestamp.
“live” UI never presents a two-hour snapshot as live.
Synthetic demo data is visibly labeled.
Privacy page corresponds to actual code.
Session projection excludes passwords/secrets.
Unknown UI capability is rejected.
Bot cannot execute arbitrary JS through capability API.
Historical ClickHouse read permission does not imply product mutation authority.
Measure E with E, while keeping system telemetry separate.
Pipeline metrics:
accepted_events_total
rejected_events_total
accepted_batches_total
worker_ingest_latency_ms
queue_depth
oldest_unflushed_event_age_seconds
flush_runs_total
flush_failures_total
flush_duration_ms
flush_events_total
flush_bytes_total
clickhouse_resume_latency_ms
clickhouse_insert_latency_ms
clickhouse_active_minutes_day
clickhouse_compressed_bytes_event
posthog_pending
posthog_delivery_success_rate
amplitude_pending
amplitude_delivery_success_rateBusiness/event metrics:
human_pageviews
semantic_events_per_session
signup_conversion
source_to_signup
portfolio_cross_domain_journeysKeep infrastructure telemetry distinct from product semantic events.
This is a deliberate learning project.
Build a small internal cost/efficiency panel showing:
analytics events/day
analytics events per human pageview
browser batches/day
events/batch
bytes/batch
Worker requests/month projection
Worker CPU-ms/month projection
Queue messages/month projection
Queue operations/month projection
ClickHouse:
flushes/day
active minutes/day
events/flush
bytes/flush
compressed GB
storage GB
estimated compute cost
estimated storage cost
estimated $ / 1M semantic eventsDo not rely forever on spreadsheet estimates.
We want to learn the true operating curve before scale.
Treat this as a hypothesis to falsify.
At current traffic:
Cloudflare CDN HTTP requests may be <700k/day.
This does not imply 700k analytics events/day.
Browser microbatching should keep Worker requests far below raw event count.
Queue microbatches should keep Queue operation volume modest.
ClickHouse two-hour flushes should create about 12 scheduled wake opportunities/day rather than continuous ingestion.
Potential early target:
Cloudflare Workers/Queues low single digits to low tens/month
ClickHouse Basic single digits to low tens/month if sleep behavior is favorable
Total core E plane target roughly $10-$30/monthAgain: measure real active minutes before claiming this publicly.
If ClickHouse is repeatedly kept awake by:
background work;
dashboard reads;
vendor fanout queries;
badly structured jobs;
fix the architecture before accepting a permanent 24/7 compute bill.
Use a few meaningful views from day one.
Examples:
pageviews_hourly
events_hourly
signups_daily
conversions_by_source_daily
sessions_daily
bot_interventions_daily laterDo not create dozens of speculative views.
For normal E web rendering, write compact summary snapshots after each 2-hour flush.
The homepage/app should display:
Last analytics refresh: 10:02 UTCor equivalent.
Freshness is a product property and should be visible.
Build a synthetic-but-realistic fixture corpus and benchmark:
-- one session journey
events for session_id over 24h
-- one user journey
events for user_id over 30d
-- site traffic
page_viewed by site over 30d
-- conversion
signup_completed by source_domain over 30d
-- simple funnel
pricing_viewed -> signup_started -> signup_completed
-- cross-domain
source domain -> destination -> signup
-- behavioral precursor
events preceding subscription_started
-- future
bot_help_accepted -> signup_completedRecord:
rows scanned
elapsed
memory
cold vs warm
candidate ORDER BYUse the results to choose schema, not intuition alone.
inspect current codebase;
inspect DNS/Cloudflare;
inventory analytics;
produce conflicts/findings;
confirm domain routing.
e.computedriven.com marketing shell;
docs/architecture/privacy placeholders;
CI/deploy;
no fake live data.
e-schema;
browser SDK;
first-party route proof on one ComputeDriven site;
validation/redaction tests.
canonical e-events Queue;
producer;
microbatching;
paid retention configured;
no continuous push consumer.
Basic service;
roles/users;
candidate schemas;
benchmark;
final v1 schema/migrations.
HTTP pull consumer;
Cron flush Worker;
idempotency;
backlog/failure tests;
measure scale-to-zero behavior.
post-flush summaries;
freshness timestamp;
public E dashboard does not wake ClickHouse.
PostHog;
Amplitude;
retries/delivery status;
canonical event ID mapping.
add first-party /_e route to sites incrementally;
compare Cloudflare traffic to human pageviews;
build cost-learning dashboard.
Only after event system is stable:
Durable Objects;
WebSockets;
chat;
SessionProjection;
typed UI capability protocol;
Super bridge.
Do not add:
Kafka;
self-hosted ClickHouse cluster;
R2 as a required hot-path dependency;
custom replay engine;
arbitrary browser-control agent;
full BI builder;
fingerprinting;
third-party-cookie hacks;
ClickHouse as product transaction DB;
second analytics Postgres;
a Phoenix backend unless the current codebase demonstrates a concrete need that Workers/Durable Objects do not satisfy;
24/7 ClickHouse queries merely to make a dashboard look “real-time.”
R2 can be added later for:
cold archive
Parquet exports
reprocessing
training/research corpora
long retentionwhen those jobs are real.
Before implementation, return answers to:
What current code already overlaps this system?
Is e.computedriven.com already present anywhere?
Which current site should be the first instrumentation target?
Can all owned domains expose /_e/* via current Cloudflare setup?
What shared package location best fits e-schema and e-sdk?
Which existing user/account IDs are safe analytics identifiers?
What current consent/privacy mechanism should E reuse?
What will be required to deploy separate dev/staging/prod Workers/Queues?
What is the simplest robust way for the scheduled Worker to pull the HTTP Queue under current Cloudflare APIs?
What duplicate/idempotency semantics can be proven for:
browser retry;
Queue redelivery;
ClickHouse insert retry;
PostHog retry;
Amplitude retry?
Which ClickHouse ORDER BY candidate wins our representative benchmarks?
Does ClickHouse Basic actually scale to zero as expected under our workload?
What active-minutes/day result do we observe with a 2-hour cadence?
Does any normal e.computedriven.com page accidentally wake ClickHouse?
Checked 2026-08-23.
Cloudflare:
Workers pricing
https://developers.cloudflare.com/workers/platform/pricing/
Cron Triggers
https://developers.cloudflare.com/workers/configuration/cron-triggers/
Queues overview
https://developers.cloudflare.com/queues/
Queues pricing
https://developers.cloudflare.com/queues/platform/pricing/
Queues limits
https://developers.cloudflare.com/queues/platform/limits/
Pull consumers
https://developers.cloudflare.com/queues/configuration/pull-consumers/
Queue configuration / retention
https://developers.cloudflare.com/queues/configuration/configure-queues/
ClickHouse:
Cloud pricing
https://clickhouse.com/pricing
ClickHouse Cloud
https://clickhouse.com/cloud
Official JS client
https://github.com/ClickHouse/clickhouse-js
Important currently verified facts:
Workers Paid includes 10M requests/month and then charges per additional million.
A Worker Cron Trigger can run scheduled jobs; triggers execute in UTC.
Queue pull consumers are intended for controlled consumption from upstream infrastructure.
Paid Queue retention can be configured up to 14 days.
Pull batch size currently supports up to 100 messages.
ClickHouse Cloud Basic is explicitly positioned for starter/testing projects.
ClickHouse Cloud advertises scaling unused compute down to zero.
The official @clickhouse/client-web client is compatible with Cloudflare Workers.
Do not freeze provider prices into product behavior. Prices are planning inputs only.
Unless codebase review produces a material contradiction, implement toward:
PUBLIC PRODUCT
e.computedriven.com
|
┌──────────────┼──────────────┐
│ │ │
marketing docs app
|
cached summaries
|
v
historical ClickHouse views
FIRST-PARTY EVENT PATH
computedriven.com/_e/v1/events
opensentience.org/_e/v1/events
...owned sites...
|
v
cd-e-ingest Worker
|
v
e-events Queue
|
buffered for ~2 hours
|
v
cd-e-flush Worker
scheduled HTTP pull
|
v
ClickHouse Cloud
|
┌───────┼────────┐
v v v
E views PostHog Amplitude
LATER LIVE PLANE
|
v
Durable Objects
|
WebSockets
|
SessionProjection
|
v
SuperThis deliberately gives E its own scaling and failure domain from the first real event.
The initial goal is not maximum scale.
The initial goal is to learn the correct scale architecture early:
edge acceptance -> durable buffering -> analytical batch ingest -> cheap historical queries -> live state somewhere else.
Break it while it is small. Measure everything. Preserve event identity. Fail without losing evidence.