System-III-Router
A learned regulator that decides how hard to think, built from
Critique of Agent Model (arXiv:2606.23991,
Xing, Deng, Hou).

Cumulative pseudo-regret over a 2,000-task stream (lower is better). The
Configurator (learned, System III) hugs the oracle floor; every fixed
deliberation depth pays a standing regret tax. Regenerate: python scripts/run_demo.py.
The idea
The paper's central distinction is agentic vs agentive: today's agents are
agentic — competence lives in external scaffolding (fixed tools, fixed
workflows) — not agentive, where the agent regulates itself. Its most concrete
proposal is System III, the Configurator: a learned controller that, each
step, decides how much to deliberate instead of always running a fixed depth.
By Theorem 3, fixed-depth deliberation is provably wasteful — easy problems need
none, hard ones need a lot — which is the gap a learned regulator fills.
This repo makes System III runnable on math reasoning. The "arms" the regulator
chooses among are deliberation modes, mapped onto the paper's cognitive stack:
| mode |
paper |
what it does |
cost |
direct |
System I (Actor) |
answer, no scratchpad |
cheapest |
cot |
System II (light) |
one chain of thought |
medium |
plan_verify |
System II (Simulative Planner) |
decompose → solve → self-verify |
most |
It is, underneath, a cost-aware contextual bandit: reward = correctness − λ·tokens,
metric = cumulative regret over a streaming task sequence. The Configurator
keeps per-difficulty success/cost statistics and learns "easy → think less,
hard → think more". That online statistic update is the fast loop of the
paper's fast-slow learning (Theorem 1): revising the self-model within the
round is what drives regret below any fixed policy.
Quickstart (offline, free, no API key)
pip install -r requirements.txt # or just: the demo needs only matplotlib
python scripts/run_demo.py # 2,000-task synthetic stream
strategy accuracy avg_reward avg_tokens cum_regret
Fixed: direct (System I only) 64.4% 0.632 25 286.5
Fixed: cot (System II light) 83.0% 0.751 158 42.6
Fixed: plan+verify (always deep) 90.1% 0.665 472 208.2
Configurator (Thompson, learned) 86.3% 0.770 186 8.3 <- learned
Oracle (zero-regret floor) 87.2% 0.776 194 0.0
>> Learned self-regulation cuts cumulative regret by +80.5% vs the best fixed depth (cot).
Read the two stories in that table:
- Regret. The learned Configurator (8.3) nearly matches the oracle (0.0) and
beats every fixed depth — the paper's claim that self-regulation must be
learned, not fixed, reproduced in miniature.
- Efficiency. It reaches near-
plan_verify accuracy (86% vs 90%) at ~40%
of the tokens (186 vs 472), and beats cot on both accuracy and reward.
"Deliberate only when it pays" is the whole point.
How the code maps to the paper
| Module |
Role in the C→A→F→C loop |
Paper concept |
src/tasks.py |
Context: task → difficulty key |
the context the regulator conditions on |
src/modes.py |
the three deliberation depths |
System I / II machinery |
src/experience.py |
per-(difficulty×mode) Beta + token stats |
fast self-model update (Thm 1) |
src/configurator.py |
Action: pick a depth |
System III, the Configurator |
src/providers.py |
run the chosen depth (mock or live) |
the solver |
src/grading.py |
Feedback: correctness − λ·tokens |
the reward |
src/loop.py + src/eval.py |
streaming regret evaluation |
performance / efficiency / growth |
src/plan_decoder.py |
Action by generation: text → deliberation plan, no featurizer |
SkillComposer core (2606.32025) |
src/plan_configurator.py + src/bottleneck.py |
the featurizer-bottleneck experiment |
— |
Run live against a 9router gateway on Vi-GSM8K
cp .env.example .env, fill in OPENAI_API_KEY, OPENAI_BASE_URL,
SYS3_SOLVER_MODEL, then:
python scripts/run_live.py 30 # Vi-GSM8K from the Hub
python scripts/run_live.py 30 --jsonl data/vi.jsonl # a local export
python scripts/run_live.py 30 --hf openai/gsm8k # plain GSM8K
One fixed solver model answers every task; the Configurator only changes how
hard it thinks. With no difficulty labels there is no oracle regret live — the
result is the accuracy-at-lower-token-cost frontier on real problems.
Does routing actually pay? (cached A/B)
scripts/run_cached_eval.py runs every (task, mode) once, caches the outcomes,
then replays the Configurator over the cache for free — recovering a true oracle
and real regret, and letting you A/B featurizers at zero API cost:
SYS3_SOLVER_MODEL=cc/claude-haiku-4-5 python scripts/run_cached_eval.py 24
What 24 matched Vi-GSM8K tasks show — the answer depends on solver strength:
| solver |
best fixed |
learned Configurator |
verdict |
| haiku (weak) |
cot regret 5.96 |
cot_steps feat → 5.28 |
routing beats fixed |
| opus (strong) |
cot regret 0.74 |
every feat → 0.74 |
routing can't fire |
For the weak solver the hard tail genuinely needs plan_verify, and an
execution-grounded difficulty feature finds it. For the strong solver, no
surface feature predicts which problems it one-shots — so a hand-featurized
bandit is stuck, and only a learned policy (see train/) can capture the
gain. The dataset's own difficulty flag is a curation filter (near-constant), so
it does not help — a clean negative result.
The featurizer is the bottleneck — a generative fix (SkillComposer)
The cached A/B hints at it; this makes it explicit. A bandit's ceiling is often
not the learning rule but the featurizer that turns a task into the key it
conditions on. src/plan_decoder.py ports the core of SkillComposer
(arXiv:2606.32025) to the depth axis: instead
of featurize → bucket → bandit, a tiny constrained autoregressive decoder
(~a few thousand params, numpy, trained by SFT) generates the deliberation plan
straight from the task text — no hand-built featurizer.
python scripts/run_plan_decoder.py
On a diagnostic stream where difficulty is signalled by a content word while
digit-count and length are held constant, count-based featurizers go blind — they
bucket every task as "easy":
strategy accuracy avg_tokens cum_regret
Bandit (featurizer=proxy, blind) 84.0% 151 47.9
Bandit (featurizer=oracle label) 87.2% 182 17.1
PlanDecoder (SFT, reads text) 87.5% 180 0.0 <- learned
Oracle (zero-regret floor) 87.5% 180 0.0
Two readings:
- Bottleneck. Same bandit, swap the featurizer (blind proxy → oracle label)
and cumulative regret jumps 2.8× (47.9 → 17.1). The learner never changed —
the featurizer was the ceiling.
- Fix. A decoder that reads the text recovers oracle routing (0.0) with no
featurizer at all: feature engineering replaced by generation. On the original
stream (where the proxy is already decent) it still closes the residual 1.6× gap
without regressing — so this isn't a cherry-picked stream.
The same decoder emits genuine multi-step plans (subset + count + order), which is
where it earns its keep on super-agent's skill axis; on the single depth
choice here the plan is length 1. A frozen pretrained encoder (Qwen3-Embedding in
the paper) is the drop-in upgrade for messy real text.
Roadmap — toward a trained Configurator (the agentive step)
The bandit above is the externalized, training-free System III. The paper wants
the regulator inside the model. The next build keeps everything here as the
eval harness and adds a learned policy:
- SFT the mode head. Use the oracle labels (cheapest mode that solves each
problem) as targets: input = problem, output =
<mode> token + solution.
- GRPO the regulator (your existing Qwen3-4B pipeline). One policy emits the
mode then the answer; reward =
correctness − λ·tokens — the same objective,
now shaping the weights instead of a Beta posterior. The model learns to spend
test-time compute only where it lifts accuracy.
- Measure growth. Re-run this regret harness on held-out Vi-GSM8K; the win
is the learned policy moving from the
cot-baseline point toward the oracle
frontier as training proceeds.
Tests
python -c "import pytest, sys; sys.exit(pytest.main(['-q']))"
25 tests lock in the behaviors, including the central claim
(test_learned_beats_every_fixed_depth_on_regret — the learned Configurator posts
lower cumulative regret than any fixed depth) and the featurizer-bottleneck fix
(test_featurizer_is_the_bottleneck_and_plandecoder_fixes_it).