Skip to main content

Results, including the inconvenient parts

The primary metric is MRR@10 on the prefix-v2 search-query task. all uses a transductive closed candidate vocabulary; train contains train-observed targets only.

Trace configPoolCoverageResetStateful adapterPrior + residualSupport proxy
Kimiall1.00000.33400.37890.45590.4132
Kimitrain0.76830.29790.33000.45420.3775
GLM-5.1all1.00000.32670.36550.53860.5522
GLM-5.1train0.79170.29330.32410.53700.5273

The stateful adapter improves over reset in all four rows. The explicit prior terms are larger than the state-only gain, which is why the project does not present the full improvement as “memory.”

Leakage and conversion audit

The canonical conversion contains 57,142 next-argument samples with zero temporal-order violations. All 212,620 stored history entries preserve observed tool responses. These checks establish causal availability; they do not establish semantic generalization.

Tool-heldout matrix

Across 2,676 BGE-M3 heldout samples and nine tools:

  • maintained state versus reset: +0.0159 MRR@10, session-clustered p=0.0001;
  • support proxy versus maintained state: -0.0032 mean delta;
  • three tools lose under support gating.

On the stricter train-only heldout audit, coverage is 23.54%, state improves by +0.0055 (p=0.0031), and the support delta is +0.0020 but non-significant (p=0.2904).

No-loss support claim rejected

Support gating can help aggregate in-distribution ranking while hurting cold heldout tools. Report the worst-tool losses and confidence tests, not only mean MRR.

URL negative control

Maintained state underperforms reset on the Kimi URL task. URL arguments are strongly affected by literal repetition and normalization, so URL results are treated as a negative control rather than clean state-memory evidence.

External TAU boundary

Three session-split TAU-bench seeds preserve temporal validity and response coupling, but train-only target coverage averages 11.92%. Ranking improvements cannot retrieve a target absent from the candidate vocabulary. Candidate generation is therefore the binding external bottleneck.

Rewrite-model refresh

MiniMax-M3 and GLM-5.2 rewrites were evaluated as query-rewrite baselines, separate from the state-adapter claim. Their metadata, paired comparisons, and provider settings are tracked in the rewrite matrix artifacts; they do not replace the frozen-encoder core result.

Machine-readable sources

Read the evidence contract before citing a result outside its evaluated boundary.