Rust SMB Experiment with Qwen3-Coder-Next and pi
Graydon Hoare had an interesting idea for an LLM based coding experiment. There are a lot of folks working on coordination between multiple agents, performing different roles on the same code base. However, most of these agents are essentially different prompts to act as different roles, doing design, writing code, reviewing code, etc, but all using the same underlying model.
Rather than dealing with the complexities of coordination, Graydon wondered if you could do all of this in a single session, asking the LLM to behave as different personas at different times. He wrote a prompt that would do development in phases, of design, writing out skeleton implementations, prototyping, evaluating prototypes, implementation, review, fixing issues, then simulating what would happen if you tried to build, test, and run it, and fixing any simulated issues found.
His writeup, and results of running this on Claude Opus 4.6 and Claude Code, are available here: https://github.com/graydon/rust-smb2
He happened to post this at just the time that I had finished setting up my local LLM environment. I'm an LLM skeptic, I don't like the idea of handing all of our coding ability over to a few big Silicon Valley companies, and I find it hard to figure out how we can safely write code if we let LLMs churn out code faster than we can review it. But its clear that capabilities have improved dramatically over the past year, so it's time to do some experimentation with it, and I needed a new laptop anyhow so I picked up a Strix Halo laptop with 128 GiB of unified RAM, which allows running decently capable open weights models locally.
Local LLM Setup
There have been some quite capable open weights models released recently. I decided to try Qwen3-Coder-Next, a coding oriented space efficient MoE model with 80B total parameters, 3B active. I'm using the unsloth/Qwen3-Coder-Next-GGUF:UD-Q6_K_XL quantization to start with (a quantization reduces the precision of parameters, trading off a little bit of accuracy for significant space savings; in general, 6 bit quantizations are supposed to not really lose very much performance at a considerable space savings over the full 16 bit training precision); this is just the first one I chose to try, I haven't really compared models yet.
I'm running this on llama.cpp.
There were some of the usual problems you'd expect getting local models set up; fiddling with memory allocation between CPU and GPU, I still haven't been able to get ROCm to work, it crashes somewhere within one of the ROCm libraries, but I got it to work with Vulkan.
I decided to use pi as the agent; I'm not a fan of proprietary dev tools, so Claude Code was out. I tried Codex as that's open source, but the latest version used APIs not yet implemented by llama.cpp. OpenCode is commonly used, but I find it a little sketchy that it phones home so much even when using it fully locally. Pi seems like a nice minimalist agent design, so I decided to try it out. Was quite easy to configure it.
I'm running everything on Fedora Silverblue, an immutable distro, so everything runs in containers. Here's my justfile I use for launching them.
build-pi-image:
podman build -t pi -f pi.Dockerfile .
pi dir:
podman run -v ~/.pi/:/root/.pi -v {{ dir }}:/code --security-opt label=disable --rm -it pi
llama-cpp:
podman run -p 8080:8080 --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined --security-opt label=disable \
--rm -it -v ~/.cache/llama.cpp/:/root/.cache/llama.cpp/ ghcr.io/ggml-org/llama.cpp:full-vulkan --server -hf unsloth/Qwen3-Coder-Next-GGUF:UD-Q6_K_XL \
--jinja --ctx-size 262144 --seed 3407 --temp 1.0 --top-p 0.95 --min-p 0.01 --top-k 40 --fit on --port 8080 --host 0.0.0.0
And here's pi.Dockerfile:
FROM buildpack-deps:latest
ENV NODE_VERSION=25.6.0
ENV NVM_DIR=/root/.nvm
ENV PATH="/root/.nvm/versions/node/v${NODE_VERSION}/bin/:/root/.cargo/bin:/root/.local/bin:/root/.local/share/pnpm:${PATH}"
RUN curl -LsSf https://astral.sh/uv/install.sh | sh &&\
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash && \
. "$NVM_DIR/nvm.sh" && \
nvm install ${NODE_VERSION} && \
nvm use v${NODE_VERSION} && \
nvm alias default v${NODE_VERSION} && \
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain stable -y
RUN cargo install fd-find && cargo install ripgrep
RUN npm install -g @mariozechner/pi-coding-agent
RUN mkdir /code
WORKDIR /code
CMD ["pi"]
And my models.json for pi:
{
"providers": {
"llama.cpp": {
"baseUrl": "http://host.containers.internal:8080/v1",
"api": "openai-completions",
"apiKey": "llama",
"models": [
{
"id": "unsloth/Qwen3-Coder-Next-GGUF:UD-Q6_K_XL",
"name": "Qwen3 Coder Next (local)",
"reasoning": false,
"input": ["text"],
"contextWindow": 262144,
"maxTokens": 65536,
"cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 }
}
]
}
}
}
This is literally the first time I've run anything on this setup, so nothing is particularly optimized; this is just the first setup that works.
Results of running the test
I just had an empty repo with the AGENTS.md taken from Graydon's test, and gave Pi the prompt "Build an SMB server in Rust that runs on a unix system and provides file service that an smbclient can use".
It cranked away for a while; I left it running overnight, and at some point my leptop went to sleep, so I'm nto sure exactly how long it took.
It went through all of the phases described in Graydon's AGENTS.md, writing each one out to a separate Markdown file which is nice. It then ignored the instruction to stop after writing all of the code out, and did try to build and run it and fix the problems that it found.
I note that it occasionally had trouble with tool calling; the biggest one was that it decided to use JSON for a config file, but could not figure out that it needed to quote the JSON within the JSON of the tool call. It eventually gave up and used a Bash tool call with cat to write out the file.
I have not really looked at the code yet, or tried it out, since it's a Monday morning and I need to get to work. I've just glanced over everything, wrote up this README.md and a .gitignore and committed it, and pushed it.
Anyhow, I think it was an interesting experiment that Graydon did, and it was a good first stress test of my local LLM setup.