A structural metric cannot distinguish a stable world from a stopped one
The degradation that matters in long sessions is not visible corruption. It is the world ceasing to respond while each frame remains individually well-formed. We report two runs below under identical input: one continues to evolve, one does not.
The stopped run scores 0.181 on structure against 0.171 for the responsive one. Ranked on structure alone it is the better of the two. The distinction appears only in the second quantity, where the two differ by a factor of eight (0.713 against 0.092).
We therefore treat long-horizon behaviour as a conjunction: structure must not collapse and inter-block motion must not decay. Results previously accepted on the structural criterion alone were re-evaluated under both, and we report the revised conclusions throughout this page.
Calibrating the structural metric against decoded frames
A metric is only useful if its range corresponds to something observable. We took the blocks at the extremes of the structural score and decoded them at full quality: at 0.035 the frame contains saturated white regions with no recoverable geometry; at 0.33 the character, wall texture and depth ordering are all resolved. The scale therefore spans presence and absence of scene content, which is the property the downstream results depend on.
Restructuring the gradient method to a constant memory cost
The published formulation notes that a direct differentiable-cache implementation exceeds available memory. Ours reproduced that at 22.9GB against a 24GB budget. Splitting the step into a rollout under no_grad, followed by a single differentiable reconstruction under a static causal mask, returns the whole-step peak to the baseline value. The mechanism is that only one forward pass carries gradient, so the sharded optimiser can reshard between units as it normally does.
Whole-step peak: 10.11GB, equal to the baseline with the method disabled.
Marginal memory cost of the method: ≈0. Measured on the same step, same batch, same shard configuration
The quality result from the same runs is negative. Across 5 seeds × 220 blocks the four metrics show no significant difference, with the point estimates slightly favouring the baseline; training loss moves from 0.2209 to 0.2207 between the first and last quarter. We report the memory result and the quality result separately because they are separate claims, and only the first is supported.
Five interventions on block latency, four of which are net negative
Each entry below was implemented and timed on the complete pipeline rather than on the affected kernel in isolation, because the two disagree. Values are the change in block latency against the unmodified configuration.
Current block latency is 672ms. A game-grade budget is 33ms.
Factor of 20. Our own near-term target of 300–400ms remains a factor of 2 away. These cards do not support FP8
What this table supports is narrower than “latency is a hardware problem”: within the five interventions we implemented, none remains that pays for itself, and the resolution result locates the cost in depth rather than sequence length. That is consistent with a hardware-bound regime but does not establish it. The remaining lever we consider credible is a smaller backbone, which would require redoing distillation and action conditioning, and which we have not yet tested.
Motion magnitude decays after block 7; both candidate mechanisms are rejected
Inter-block motion is flat for the first six blocks and then declines monotonically, accumulating to −45% by block 60. Two mechanisms were plausible enough to test directly. Both are inconsistent with the measurements.
Controlled comparison against an equivalent checkpoint trained without it, single variable: slopes −0.039±0.007 and −0.032±0.018 over 5 seeds. The difference lies within the seed spread, and the point estimate favours the distilled variant. The premise for retraining does not hold.
The window size divides to exactly seven blocks, which matches the onset. Widening it leaves the onset unchanged: three window sizes agree position by position across the first seven blocks, and the two larger ones remain identical to each other afterwards, indicating a separate constraint is binding.
We are not proposing a mechanism until one survives a test.
Both candidates above were rejected against our own data. An untested explanation that sounds correct is the more expensive outcome, because it ends the search
Ten hypotheses, the measurement that rejected each, and its cost
Each of these was reasonable enough to pursue, and each was rejected against our own data. Bar length is the effort spent before the rejection. Where a rejection also invalidated earlier conclusions, that is stated. Open a row for the measurement.
Every evaluation — guards, perceptual validation, the memory work, drift exposure, dynamic sink, cache rollback — was run without the flag, and therefore with the sink disabled, while the shipping build hardcodes it on. Under the correct configuration, 5 seeds × 400 blocks all had positive slope and never collapsed. The six rounds were not measuring a real defect. The check that now precedes any evaluation is a line-by-line comparison of experiment configuration against shipping configuration; in this instance a single startup log line would have shown it.
Collapsed blocks got 1.5–14× worse. Distillation makes the student match the teacher given the context; when the context is accumulated drift, the teacher’s output is self-consistent with that drift — so it teaches the model to degrade gracefully. The correct form is corrupted input with a clean target; we had built corrupted input with a self-consistent one.
5 seeds × 220 blocks: four metrics, no significant difference, directionally slightly worse. Training loss moved 0.2209 → 0.2207 between the first and last quarter. The memory problem was solved. The algorithmic one was not.
After fixing a seed that was being silently overwritten, the spread across seeds within one configuration (1.13–3.73) turned out to be far larger than the difference between configurations. Every earlier ablation had been reading noise, and the mechanism story built on top of it went with it.
Net +126ms (540 against 414). The bare integer matmul really is 2.1–3.7× faster at the real shapes, but hand-written quantise/dequantise traffic makes the layer slower overall; the fused kernel is 2.78× faster on the feed-forward yet demands a dtype the pipeline does not use, and converting across 305 layers eats the entire gain. An isolated GEMM benchmark is not the cost of the substitution.
Controlled comparison, single variable. Slopes −0.039±0.007 and −0.032±0.018 — entirely inside seed noise, and directionally better with it. The premise for a full retrain disappeared.
Tempting: 319ms against 508ms. The output is NaN — the motion metric reaches 6.2e6; one setting down is all zeros. All three settings completed without error and produced 200 output files with the expected latency figures. Nothing in the logs distinguishes the working setting from the two that produce no image, which is why the acceptance criterion for this class of change is a decoded frame rather than a completed run.
The arithmetic matched perfectly. Widening the window moved nothing: three sizes identical value by value across the first seven blocks, and the two larger ones identical to each other afterwards — something clamps them, and all fall close to frozen.
At the lower resolution it reliably turned the slope from −0.023 to +0.009, reproduced across two seeds. At the delivery resolution the two seeds went in opposite directions. A hyperparameter tuned at one resolution does not transfer; carrying it over buys zero gain and a 13% detail cost.
Decoding the same latents with both decoders and comparing directly, the lightweight decoder is the sharper of the two. This contradicted an assumption we had held internally for several months. The dominant factor is resolution: the two settings under comparison differ by a factor of 2.5 in pixel area.
Quality degraded across sessions, and the cause was retained state rather than the model
Users reported progressive blurring within a session and occasional content from a previous session, while offline runs at identical resolution and weights showed neither. Isolating by first-frame size across three consecutive sessions reproduced the degradation deterministically, which located the fault outside the model. Three caches were surviving session boundaries.
before — first frame degrades each session
after — byte-for-byte identical
102383 / 102383 / 102383 bytes
| attention cache | reset zeroed the index but left the tensor contents; the sink and the rolling window then read slots this session never wrote |
| cross-attention cache | only the initialised flag was reset |
| decoder temporal cache | the culprit — streaming decode deliberately keeps temporal state across calls, which is correct, but nothing cleared it between sessions: the second player’s first block was decoded as a continuation of the first player’s frames |
First-frame size across three sessions: 102KB / 62KB / 67KB before, and byte-identical after.
This also accounts for the standing discrepancy between offline and live quality: an offline run starts a new process, and therefore an empty cache, on every invocation
A second effect appears only under interaction: a zero action vector is not a stop command. After all inputs are released the model continues to propagate motion from context, measured at 1.181 in the first block after release against 1.235 while held, and requires 6–8 blocks — roughly five seconds — to settle. Control is unavailable during that interval. We currently suppress it at inference time by freezing the world when no input is held; this is a mitigation at the serving layer and not a correction in the model, and we record it as such.
What transfers from published work, and what had to be measured here
Available from published work
- An open-source base model
- Distillation, low-rank adaptation, attention sink, key-value caching
- A working single-session demonstration
Established by measurement in this work
- A two-part criterion, calibrated against decoded frames, that separates a stable world from a stopped one
- Ten rejected hypotheses with the measurements that rejected them, including one that invalidated six prior optimisation rounds
- The failure modes of continuous serving: three caches crossing session boundaries, momentum after input release, and a granularity setting that completes without error while producing NaN
- A characterised operating point on consumer hardware, with the memory and latency boundaries located rather than estimated
- One decay that remains unexplained, with both leading mechanisms ruled out
The position behind this work is that local hardware is the substrate of control: a model that can only be reached through someone else’s datacentre belongs to whoever owns it, and moving inference onto the user’s own device is the concrete step that changes that. It is a research problem and an engineering problem before it is anything else, which is why this page is a set of measurements rather than an argument. Against that objective, the accounting here reads: memory is no longer the binding constraint; latency is, by a factor of 2 against our own target and 20 against a game-grade budget, with no algorithmic lever found that closes it; and one decay remains unexplained. The next test is a smaller backbone.