Skip to main content

From chronological traces to an auditable score

The pipeline is designed so that every quantity used at step (t) is available before the target action at step (t). Conversion, representation, scoring, and evaluation each enforce a different part of that contract.

1. Build a causal prefix​

For every tool call, the converter emits the current query, the target tool argument, and only the actions and responses observed earlier in the same trace. The invariant is:

query_message_index < action_message_index

Responses are coupled to the action that produced them. Parallel calls share the same observed prefix, and session identities are preserved for clustered inference.

2. Keep the encoder frozen​

Let q_t ∈ R^d be the normalized BGE-M3 embedding of the current query and c_j ∈ R^d the fixed embedding of candidate j. The baseline score is:

s_reset(t, j) = q_tᵀ c_j

History actions are encoded once, gathered by index, and aggregated causally. A compact adapter produces a state residual r_t, giving:

s_state(t, j) = norm(q_t + r_t)ᵀ c_j

Repeated targets use a multi-positive contrastive objective, so identical correct arguments are not treated as in-batch negatives.

3. Make target frequency explicit​

For tool-conditioned train counts n_g,j, the prior uses centered log counts:

p_g,j = log(n_g,j + α) - mean_k∈C log(n_g,k + α)

The combined score is:

s(t, j) = s_state(t, j) + λ_t p_g(t),j

The gate λ_t is reported as its own method. It is never folded invisibly into the claim that state helped.

4. Evaluate two candidate worlds​

Every main result distinguishes:

  • all_splits: a transductive closed vocabulary built from all splits;
  • train_split: candidates observed in training only.

Coverage is reported beside ranking quality. A missing target is a miss, not a row silently removed from evaluation.

5. Stress the claim​

The evidence package includes tool-heldout splits, a URL negative control, cross-configuration transfer, session-clustered significance, rewrite-model comparisons, and TAU-bench external boundary runs. See Results for the conclusions each check permits.