Appendix G Proofs

Finiteness. By Assumption 2, $p^{\star }$ (or its best computable approximation, in the M-open case) lies in the support of the class entertained by $g$, so $q_E$ is absolutely continuous with respect to $p^{\star }$ on the relevant support and .

Monotonicity. For the conjugate update (13), the posterior predictive is the Bayes-optimal predictor under log-loss. By Lemma 3 the expected divergence is summable and cannot persistently increase; for a single well-specified conjugate family it is monotone non-increasing at each step, since absorbing $e$ only sharpens $\mathrm{Beta}({\alpha }_h,{\beta }_h)$ toward the realized frequency. Hence $𝔼D_{\mathrm{KL}}(p^{\star }\parallel q_{E\cup e})\le D_{\mathrm{KL}}(p^{\star }\parallel q_E)$.

No narrative analogue. An NWM emits $\tilde{q}$ directly (or estimates it from samples) with no conditioning operator: “updating” is re-prompting the language model $L$, a map not constrained to be Bayesian, so $D_{\mathrm{KL}}(p^{\star }\parallel \tilde{q})$ may increase with new evidence; and when the predictive is read from sampled points, its law can depend on prompt framing, so the divergence need not even be well-defined. It does not inherit the guarantee. ∎

Model the NWM as a fixed language model $L$ equipped with an update operator $U_L$ that maps a state—context tokens, or a knowledge base together with a retrieval policy—and a new evidence item $e$ to a new state, from which the implied predictive $\tilde{q}$ is read by decoding. Write $B_e$ for the exact Bayesian update $p(\cdot \mid E)\mapsto p(\cdot \mid E\cup e)$.

(i) Non-preservation. $U_L$ is computed by attention over a finite, compressed context (or by a bounded-recall retrieval step), and its output is the decoded next-token law—a continuous function of the prompt embedding with no constraint tying it to $B_e$. The two maps therefore agree only on a measure-zero subset of $(L,e)$: generically ${\tilde{q}}_{E\cup e}=U_L({\tilde{q}}_E,e)\ne B_e\left(p^{\star }(\cdot \mid E)\right)=p^{\star }(\cdot \mid E\cup e)$ even when ${\tilde{q}}_E=p^{\star }(\cdot \mid E)$. The discrepancy is governed by attention allocation, context-window truncation and compression, and retrieval precision, none of which implement conditioning, so propriety is not invariant under $U_L$.

(ii) Non-convergence and unbounded error. Because the per-step map is not a Bayes update, the cumulative log-loss does not telescope to $-\log$ of the marginal likelihood, the prequential bound (14) does not apply, and nothing drives the per-step divergence to zero. Concretely, the context window is finite: for any horizon there is an evidence stream whose informative items are evicted or compressed away, after which $\tilde{q}$ is independent of them. Choosing a query $Y$ whose answer depends on the evicted evidence makes $p^{\star }(\cdot \mid E)$ place mass where $\tilde{q}$ places none, so $D_{\mathrm{KL}}(p^{\star }\parallel \tilde{q})$ exceeds any prescribed $\delta$. The same holds for KB-augmentation whenever the retrieval policy misses the relevant item.

The exception. Both failures vanish only if $L$ internally represents the exact mechanism of $p^{\star }$ and its attention/retrieval routes each $e$ to that representation as a conditioning operation. Treating the realized circuit as a draw from the function class of an $|L|$-parameter network of structural complexity $K$, the subset implementing exact conditioning has prior measure decreasing in both $K$ and $|L|$: more parameters admit exponentially more circuits, among which the correctly-wired one is a vanishing fraction. The event is thus non-generic and becomes less likely as either grows.

Consequences. Two consequences are stated formally in Appendix H: enlarging $L$ does not raise the prior mass of the exception (Corollary 2), and replacing $U_L$ with a gradient step does not recover propriety either (Corollary 3). In both cases a guarantee of a proper posterior requires an explicit, verified conditioning operator external to $L$—coupling $L$ to a GWM it calls (Section 4.1). ∎

(a) is the standard delta-method variance of a smooth functional of a Monte-Carlo estimate. (b) is immediate from Pearl’s do-calculus: matching the identities is equality of the interventional distributions on $𝒜$, so the KL contribution there is $0$; the linear-in-$(1-\mathrm{CC})$ bound follows by counting violated checks. (c) is a covering-number bound: with $d_{\text{free}}=m(1-S)$ unconstrained directions and an $L$-Lipschitz $T$, the estimation variance scales with the feasible-set volume $\propto d_{\text{free}}$. ∎

Lemma 4 already gives, on the checked queries, $D_{\mathrm{KL}}(p^{\star }\parallel q)\le c_1(1-\mathrm{CC})+c_2(1-S)+c_3{(1-R)}^2$ for constants $c_i$ depending on $L$ and the battery—no faithfulness assumption used. Thus higher $\mathrm{CC}$, $S$, $R$ provably lower the divergence on $𝒜$, so on those queries $\mathrm{XQ}$ is monotonically related to $\mathrm{PQ}$.

It remains to pass from the checked queries to the full query distribution. By Assumption 1 (representative checks), the un-tested approximation error is at most an $O(ϵ)$ remainder, and $\mathrm{HtV}$ controls it: a high $\mathrm{HtV}(q)$ means few alternative models fit the same data, so by the Occam/MDL argument the certified-on-$𝒜$ model is close to $p^{\star }$ off $𝒜$ as well (the surviving-explanation volume is small). Assumption 2 keeps the divergence finite. Combining, $D_{\mathrm{KL}}(p^{\star }\parallel q)\le c_1(1-\mathrm{CC})+c_2(1-S)+c_3{(1-R)}^2+c_4(1-\mathrm{HtV})+O(ϵ)$. This bound is a sum of gaps $(1-\cdot )$, whereas $\mathrm{XQ}$ of (6) is a sum of logs; the elementary inequality $1-x\le -\log x$ on $(0,1]$ (and ${(1-R)}^2\le 1-R\le -\log R$) bounds each gap by the negative log of its attainability, so with $\overline{c}=\max \{c_1,c_2,c_3,c_4\}$,

a monotone affine lower bound, tight to first order as the attainabilities approach $1$. This one-sided relation—not a bijection—is exactly what the downstream results require: $\mathrm{XQ}$ lower-bounds closeness to $p^{\star }$, so raising $\mathrm{XQ}$ raises the guaranteed floor on $\mathrm{PQ}$, and $\mathrm{XQ}$ can never certify a prediction quality the model does not have. Equal-$\mathrm{XQ}$ models may still differ in $\mathrm{PQ}$, which the proposition’s “up to a bounded slack” already permits. Hence XQ is a valid observable proxy for the unmeasurable PQ, and the $\mathrm{CC}$, $R$, $S$ channels are results rather than assumptions. ∎

We must distinguish two quantities. The ceiling of a WM class is the attainable supremum of its components, aggregated by (6); the deployed attainment is the joint correctness the model actually realizes at its operating budget—a point on the cost curve. For a GWM the two coincide: a posterior is exact given its evidence, so the GWM sits at its ceiling at flat cost (attained $=$ ceiling). Being mechanism-pinned it is exact-by-construction on stiffness ($S=1$) and counterfactual consistency ($\mathrm{CC}=1$); its hardness-to-vary is , reflecting bounded misspecification (Assumption 2); and it has the highest residual-noise control $R$, because exact / large-sample belief propagation drives $\mathrm{se}(q)$ to near zero. For a NWM the two differ: the table reports its deployed attainment, with the structural triple $(S,\mathrm{CC},\mathrm{HtV})$ taken as the measured medians over the one-shot NWM memos (Section 3.2). Residual control is also measured at the deployed budget (the simplex-volume residual-uncertainty of Definition 11): the NWM’s one-shot $R=0.73$ sits below the GWM’s large-sample $R=0.95$, so $R$ does not cancel—the GWM leads on all four axes. The GWM’s $R$ is granted only as the NWM’s theoretical ceiling (reachable by MC-averaging many emissions). The measured CC is the two-step estimator of Appendix C, $\min ({\mathrm{CC}}_{\mathrm{stat}},{\mathrm{CC}}_{\mathrm{int}})$, combining the static internal-consistency battery with an off-grid interventional re-query of the same NWM; at the deployed budget the static step binds, so the reported CC remains a conservative estimate of the interventional ideal of Definition 9. The NWM’s theoretical ceiling is strictly below the GWM’s (Proposition 5); for all quantification we conservatively set it equal to the GWM’s, so the cost-divergence and dominance results hold a fortiori. The per-component values are:

• •

  GWM (attained $=$ ceiling) $(S,\mathrm{CC},\mathrm{HtV},R)=(1,1,0.92,0.95)$. $S=\mathrm{CC}=1$ because the probabilistic program enforces the elasticity bands and the do-calculus identities exactly (Definitions 8 and 9); encodes bounded misspecification (Assumption 2)—conditioning still leaves some explanatory slack; and $R=0.95$ is the highest residual control, from exact / large-sample belief propagation (Definition 11).

• •

  NWM (deployed, one-shot) $(0.83,0.88,0.67,0.73)$. All four are measured medians over the one-shot NWM memos—a deployed attainment, not a ceiling; the fourth, $R=0.73$, is the simplex-volume residual control of Definition 11 and sits below the GWM’s $R=0.95$, so the two arms differ on all four axes (the GWM’s $R$ is granted only as the NWM’s theoretical ceiling).

Aggregating by Eq. (6) gives, for each arm, the log-XQ (nats) and the bounded attainment $A={\prod }_i{c}_i=\exp (\mathrm{XQ})$ used as the target axis in the figures—which for the GWM is its ceiling and for the NWM is its deployed attainment. The deployed-budget quality gap is the ratio of the two attainments, which equals $\sim 2.4$.

∎

Each structural component certifies a property a mechanism-free model cannot guarantee. (CC) Without a $\mathrm{do}$-operator the interventional distribution is not computed from a causal graph, so there exist interventions $a$ for which the do-calculus identities fail and the pass fraction is . This is not merely an artifact of the present construction: by Richens and Everitt (2024), robustly passing a large interventional battery requires an approximate causal model, so a model that achieved $\mathrm{CC}\to 1$ would have implicitly learned one—and would thereby be a GWM, contradicting the premise that the narrative WM carries no explicit mechanism. (S) Without named structural parameters carrying admissible elasticity bands, at least one input acts as a free knob, so . (HtV) A free-form rationale admits a local edit fitting an alternative outcome without breaking any checkable invariant, so on a positive-measure set and . Each bound is independent of compute. Substituting the suprema into (6) gives (a sum of logs of quantities ). Residual control is reducible to the GWM’s large-sample value by MC-averaging—whether the predictive is drawn as samples or emitted wholesale—so it does not affect the structural bound. ∎

Inverting $C(x)=B$ gives ${(x_c/(x_c-x))}^{\eta }=B/c_0$, so $x_c-x=x_c{(c_0/B)}^{1/\eta }$ and $x(B)=x_c(1-{(c_0/B)}^{1/\eta })$. Since $c_0,B,\eta >0$ the correction is positive, so , $\to 0$ as $B\to \mathrm{\infty }$. The divergence of $C$ as $x\uparrow x_c$ is established by the cost-model blow-up result (Proposition 4, stated in Appendix F and proved below), which derives the output term of Eq. (8) and identifies $\eta$. ∎

Every factor $c_{i,\mathrm{G}}/c_{i,\mathrm{N}}\ge 1$ and the three structural ones are $>1$ at the deployed budget, so the sum in (19) is positive; equivalently the joint-correctness ratio ${\prod }_i{c}_{i,\mathrm{G}}/{\prod }_i{c}_{i,\mathrm{N}}>1$. With the values of Table 4 the GWM’s joint correctness ${\prod }_i{c}_{i,\mathrm{G}}=0.87$ exceeds the NWM’s $0.36$ by $2.4\times$. Closing this gap requires raising the NWM’s achieved XQ, whose per-PD cost diverges as $x\uparrow x_c$ (Proposition 6), while the GWM holds its value at the flat cost $c_{\mathrm{PD}}$; hence at any finite budget the cost-equalized comparison strictly favors the GWM. ∎

Integrating the saturating-coverage law of Lemma 1, $dx/(x_c-x)=\lambda d\nu$, yields $-\ln (x_c-x)=\lambda \nu +\text{const}$, hence

which diverges as $x\uparrow x_c$. By Lemma 2 the refinement cost is $C(x)\propto k\sim {\overline{q}}^{-\nu (x)}=\exp \left(\nu (x)\ln (1/\overline{q})\right)={\left(x_c/(x_c-x)\right)}^{\eta }$ with $\eta =\ln (1/\overline{q})/\lambda >0$, which is Eq. (11). Since Lemma 2 bounds the pass count from below by ${\overline{q}}^{-\nu }$, this is a lower bound on cost (equivalently, an upper bound on the attainment a fixed budget buys), so the leading-order blow-up is as stated. Finally the pole is impassable: by Proposition 5 the structural components are bounded away from $1$ at every budget, and reaching $\mathrm{CC}\to 1$ would by Richens and Everitt (2024) require an approximate causal model, contradicting the mechanism-free premise; hence at any finite budget and $C\to \mathrm{\infty }$ as $x\uparrow x_c$. ∎