Review date: 2026-09-16 Reviewed source: /home/travis/ProjectAmp2/computedriven commit 87f51f7c0c96fc43cefea93d5a16282a698a64bf Decision: HOLD the official competitive battery. Stage 0 is a useful correctness harness, but its current performance numbers do not compare equal measurement windows or fully comparable physical cost.
The repository was clean at the reviewed commit.
The quick five-system control suite was rerun independently. All ledger/oracle, replay, conservation, starvation-visibility and arrival-time controls held across Linux, Go, BEAM, ComputeDriven and Seastar.
The whole repository guard suite passed when build output was redirected to writable scratch storage: ALL GUARDS PASS.
The adapters use the intended native scheduling shapes: Linux fair-class threads on one CPU, Go with one P, BEAM with one scheduler, Seastar with one shard and the documented 0.5 ms task quota, and the ComputeDriven policy arm with explicit residency.
Stage 0 is honest about its retained smoke being non-evidentiary.
These checks establish semantic equivalence on drain traces and show that the harness can expose starvation. They do not validate the competitive performance scorecard.
The source writes a STOP frame at the intended deadline, but each adapter keeps counting until its own runtime schedules the reader and processes that frame. That makes the scheduler under test responsible for deciding how long it is measured.
An independent 50 ms development smoke was run on the shared host only to audit the instrument. The 2B closed-loop workload had a 100 ms offered interval after the common 50 ms start delay. The following work was recorded after t_stop_sched_ns:
| row | adapter exit after deadline | recorded samples after deadline | latest recorded sample after deadline |
|---|---|---|---|
| ComputeDriven CLOCK-AGE | 0.006 ms | 0 | none |
| ComputeDriven RR-FAIR | 0.004 ms | 0 | none |
| Linux | 0.750 ms | 4 | 0.391 ms |
| BEAM | 2.363 ms | 141 | 1.948 ms |
| Seastar | 32.080 ms | 21,165 | 32.071 ms |
| Go | 639.213 ms | 435,108 | 639.177 ms |
For Go, the corresponding B and 2B processes ran about 741 ms and 789 ms from adapter start, even though their offered intervals differed by only 50 ms. The current marginal-instruction calculation subtracts these runs and divides by the admitted-transition delta. That slope is not a cost per transition; it includes two different, runtime-dependent overruns. The retained smoke's 623 ms Go age is explained by this same defect.
Relevant implementation points are tools/fcb_smoke.py:202, fcb/go/fcb_go.go:521, and fcb/seastar/fcb_seastar.cc:176.
tools/fcb_smoke.py:46 applies exclude_kernel to instructions, cycles and the software context-switch event. Linux documents context switches as kernel events, so that configuration produces zero. The independent C6 rerun showed perf_event context switches equal to zero for every row while the same runs reported nonzero voluntary/involuntary switches through process accounting. C6 still returned held: true, because it checks availability and multiplexing but has no non-vacuity assertion for the event's value (tools/fcb_controls.py:343-381).
The same smoke then uses user-only instructions for every row in one Pareto calculation (tools/fcb_smoke.py:142-153). This omits EEVDF's kernel work while counting the Go, BEAM and Seastar schedulers in user space. A prose caveat does not make those values comparable.
The Linux kernel documentation confirms that perf_event_paranoid >= 2 disallows unprivileged kernel profiling. The perf_event_open(2) documentation also states that PERF_COUNT_SW_CONTEXT_SWITCHES is reported as a kernel event on modern Linux.
tools/fcb_smoke.py:103-121 excludes only mailbox admission refusals. It puts semantic REFUSED and BAD samples in the latency population even though the model defines progress as an admitted successful transition.
The audit reproduced this directly on ComputeDriven LONG: the row had 2,782 admitted transitions and 418 length refusals, while the latency population was 3,200. Resolution latency and progress latency are both useful, but they are different metrics and must be reported separately.
The current oldest-ready calculation also uses only the first sample for an event and, for closed-loop demand, the gap between every 64th sampled transition. That is not the maximum age before the next transition. Later intents in a multi-count event can wait without appearing in the age maximum, while sampled closed-loop gaps include 64 units of useful work.
The adapter says this plainly in fcb/cd/src/main.rs:1-38: it depends only on cd-core, recreates the kernellet's step shape, and implements CLOCK-AGE/RR-FAIR plus parking inside a standard Rust process. It does not invoke the production cd-kernellet image or its 48-byte command/32-byte outcome ABI.
That row is valid as a ComputeDriven policy/home mechanism comparison. It cannot support the stronger statement that the no-std floor/kernellet as shipped beats Linux, Go, BEAM or Seastar. The model currently calls it a complete floor + home system at docs/fcb-model.md:51, so the label exceeds what executes.
The production kernellet already has a separate real-image instrument in tools/kernellet_bench.py. The honest program is therefore two linked campaigns:
Scheduler-outcome campaign: current deterministic demand model, with the present arm renamed computedriven-policy or equivalent. This tests progress/fairness and explicit residency policy.
Kernellet-cost campaign: the actual cd-kernellet ELF and ABI against semantically equivalent transition loops. This tests the real floor's transition price and footprint.
Do not introduce a fake pipe-based “end-to-end” kernellet row and call its host IPC overhead the floor. A true home-to-kernellet integration should wait for an actual deployment boundary or an explicitly ruled ABI extension.
Replace CLOCK_REALTIME with one cross-process monotonic clock. The current source and adapters use wall-clock time (fcb/source/fcb_source.c, fcb/c/fcb.h:53-56, and fcb/cd/src/main.rs:54-63). Wall-clock adjustments must not change a benchmark latency.
Turn source lateness into a predeclared admissibility gate for official runs. It is currently reported but the smoke still calculates frontiers on late rows.
Do not infer per-entity RSS from one N=32 run and one N=64 run. The independent C6 rerun produced a negative ComputeDriven slope and a roughly 2 MiB/entity Go slope, which demonstrates noise/startup allocation rather than a stable memory price. Use repeated populations and a robust fitted slope, or report only measured peak RSS.
Tier B remains incomplete for BEAM and Seastar. Tier A can eventually support a narrow native-outcome claim after the blocking repairs. No selection/residency claim across all five systems is allowed until every Tier B arm exists and passes the same controls.
Record the Linux fair-class identity in every receipt: kernel version, SCHED_OTHER, requested and read-back slice, and sched-ext state. This review observed kernel 7.2.2-arch1-1 and /sys/kernel/sched_ext/state = disabled.
Fable owns all source changes. Codex will review the resulting commit and evidence; Codex should not implement these repairs.
Version the wire format and add a post-readiness START or ARM frame containing a common monotonic t0 and absolute deadline. Every worker must know the deadline before demand begins.
For timed traces:
A transition counts as progress only if it completed at or before the deadline.
A transition begun before but completed after the deadline is reported as inflight_at_deadline; it is not admitted progress for that window.
Reader/cleanup delay after the deadline may be reported, but it cannot add progress, samples, bytes moved or scheduling credit.
The source still sends STOP so adapters can cleanly drain control traffic and exit.
Report deadline_ns, last_counted_transition_ns, post_deadline_progress, inflight_at_deadline, and cleanup_after_deadline_ns.
Add control C7 window closure. Its non-vacuity mutant ignores the armed deadline and continues until STOP. C7 must catch it on at least Go and Seastar. Acceptance requires post_deadline_progress = 0 on every row and no primary sample timestamp beyond the deadline.
Maintain separate sets and columns:
ready_to_progress: time to the first ST_OK transition for an event;
ready_to_resolution: time to the first terminal outcome, including semantic refusal;
oldest_ready_age_max: maximum age of any actually ready Locus before its next completed progress;
max_inter_progress_gap for closed-loop demand, recorded at every transition inside the adapter rather than inferred from every-k sampling;
refusal latency and reason distribution.
For multi-count events, oldest-ready age must see the last queued intent as well as the first. Sampling may remain for replay evidence, but a sampled 64-transition gap cannot be the primary age metric.
Add control C8 metric semantics with two mutants: one counts ST_REFUSED as progress latency; the other looks only at the first intent in a multi-count event. Both must turn C8 red.
Configure hardware instructions/cycles separately from software context-switch accounting.
Do not set exclude_kernel on a software event and then interpret zero as an observation. If permissions make the event unavailable, emit null with the reason.
Add uniform process-level voluntary/involuntary switch reporting for every adapter, and retain Linux per-thread schedstat as a diagnostic.
Add an instrument non-vacuity workload that must cause switches. C6 fails if the chosen switch instrument remains zero while the control workload and independent accounting show switches.
Split frontiers. The outcome frontier may compare admitted progress and worst-Locus delay across all rows. An instruction-cost frontier may combine rows only when the counted execution domains match. With perf_event_paranoid = 2, Linux user-only instructions are a separate diagnostic and cannot rank against full user-space scheduler cost.
If kernel-inclusive Linux instructions are later enabled through a narrowly provisioned measurement launcher, record that capability and the exact event domain. Do not grant a general interpreter performance-monitoring capability.
Add control C9 counter domain. Its mutant reuses the current all-events exclude_kernel mask and must be caught.
Rename the current arm and claims so they say computedriven-policy/cd-core policy adapter until a production integration exists. Update the model, JSON, table labels and falsifiers together.
Create a separate written contract for the kernellet-cost campaign before adding code. It must use the real cd-kernellet ELF built from the reviewed source and the existing ABI/census path. It must say exactly which fixed costs marginal subtraction removes and which costs remain. Stop for Codex review of that contract before running it.
Build an official runner only after C7-C9 hold. It must:
enforce the standing dedicated-host profile and fixed sentinel gates;
alternate row order within blocks;
reject a row when source lateness exceeds a predeclared bound;
reject a whole block on a deadline, counter-domain, isolation or sentinel violation;
preserve every raw run and derive tables from them;
report confidence intervals/repeat dispersion without a synthetic score;
refuse to emit a Pareto claim when required dimensions are incomparable or missing.
Do not run that official runner in the development session.
Codex should authorize a dedicated-host Tier A scheduler-outcome battery only if all of the following hold:
C1-C9 pass with named non-vacuity mutants caught.
A short adversarial closed-loop audit shows zero post-deadline progress in all rows.
Refusal-only events appear in resolution latency and never in progress latency.
Context-switch instrumentation is nonzero on its switch-inducing control or explicitly unavailable; zero cannot silently pass.
The cross-system frontier contains only comparable dimensions.
The current ComputeDriven row is labelled as the policy adapter rather than the production kernellet.
Repository guards pass and the source tree is clean at an exact commit.
Tier B and the real-kernellet cost campaign receive separate reviews. No single winner claim combines them.
Linux EEVDF uses lag and virtual deadlines and supports requested slices through sched_setattr: https://docs.kernel.org/scheduler/sched-eevdf.html
Linux perf_event_paranoid >= 2 disallows unprivileged kernel profiling: https://docs.kernel.org/admin-guide/sysctl/kernel.html#perf-event-paranoid
PERF_COUNT_SW_CONTEXT_SWITCHES is a kernel-reported software event on modern Linux: https://man7.org/linux/man-pages/man2/perf_event_open.2.html
Go supports asynchronous safe-point preemption and signal-based suspension of user goroutines: https://go.dev/src/runtime/preempt.go
Go exposes runnable scheduler latency through /sched/latencies:seconds: https://pkg.go.dev/runtime/metrics
BEAM +S controls scheduler threads and +sbwt controls scheduler busy-wait behavior: https://www.erlang.org/docs/20/man/erl.html
Seastar documents task_quota_ms as maximum time between polls, default 0.5 ms, and documents its idle-poll behavior: https://docs.seastar.io/master/structseastar_1_1reactor__options.html
The answer to “can we benchmark now?” is split:
Correctness and benchmark development: yes. Stage 0 is strong enough to continue.
Official competitive performance: no. The current smoke cannot rank the systems because the window and cost domains differ.
Closest path to a real comparison: Fable completes this bounded stage-0.1 repair; Codex independently reruns C1-C9 and the window audit; then the dedicated-host Tier A campaign can be considered.