An evaluation framework for AI agents. Bring any benchmark, agent or capability, run it on your machines, and inspect the evidence behind every score.
Repo ThinkFlowLab/nullius, read from GitHub 8 h ago. ThinkFlowLab has not joined Take My Repo.
v0.1.1 · Hugging Face endpoints, OSWorld and machine constraints
https://hf-mirror.com from Settings → Data providers, or with HF_ENDPOINT. It reaches drivers, Original setups, MCP servers and runs on hosts; pinned files are still checked by their sha256.agent.computer/v1 port it declares, and a locked install fetches the git submodules its lock names.nullius standard run ... --units FILE runs only the units a file names; merge refuses shards whose lists differ.Thank you to everyone who built this release:
@aitianshi, @harenome and @pierreleca.
Install the latest version with uv and Python 3.12+:
uv tool install nullius-eval nullius serve
For an existing unpinned installation, run uv tool upgrade nullius-eval.
Open http://127.0.0.1:5057.
Test and CI changes: #135.
Full Changelog: v0.1.0...v0.1.1
To pin this version, use uv tool install nullius-eval==0.1.1.
Alternatively, install the attached wheel with
uv tool install ./nullius_eval-0.1.1-py3-none-any.whl.
Add --force when replacing an existing tool installation. Settings,
catalogue additions and runs stay outside the installed package;
nullius paths shows their locations.
The downloads contain the wheel, source archive and SHA256SUMS.
After downloading all three files into one directory, verify them on Linux
with sha256sum --check SHA256SUMS.
Release validation targets Linux with Python 3.12: backend and route tests,
installed commands and resources, and user-storage preservation. It does not
establish macOS or Windows support or validate every live benchmark/provider
combination. These are Python packages with a local web UI.
v0.1.0 · First release
docker context use selects.Thank you to everyone who built this first release:
@aitianshi, @harenome, @vultkayn, @yiannislamprou, @42int42 and @opPO333.
Install the latest version with uv and Python 3.12+:
uv tool install nullius-eval nullius serve
For an existing unpinned installation, run uv tool upgrade nullius-eval.
Open http://127.0.0.1:5057.
docker context use selects by @harenome in #76Test and CI changes: #17, #25, #35, #75, #82.
Full Changelog: v0.1.0rc2...v0.1.0
To pin this version, use uv tool install nullius-eval==0.1.0.
Alternatively, install the attached wheel with
uv tool install ./nullius_eval-0.1.0-py3-none-any.whl.
Add --force when replacing an existing tool installation. Settings,
catalogue additions and runs stay outside the installed package;
nullius paths shows their locations.
The downloads contain the wheel, source archive and SHA256SUMS.
After downloading all three files into one directory, verify them on Linux
with sha256sum --check SHA256SUMS.
Release validation targets Linux with Python 3.12: backend and route tests,
installed commands and resources, and user-storage preservation. It does not
establish macOS or Windows support or validate every live benchmark/provider
combination. These are Python packages with a local web UI.
v0.1.0rc2 · Package description update
Application behavior is unchanged from v0.1.0rc1.
Install the latest version with uv and Python 3.12+:
uv tool install nullius-eval nullius serve
For an existing unpinned installation, run uv tool upgrade nullius-eval.
Open http://127.0.0.1:5057.
Full Changelog: v0.1.0rc1...v0.1.0rc2
To pin this candidate, use uv tool install nullius-eval==0.1.0rc2.
Alternatively, install the attached wheel with
uv tool install ./nullius_eval-0.1.0rc2-py3-none-any.whl.
Add --force when replacing an existing tool installation. Settings,
catalogue additions and runs stay outside the installed package;
nullius paths shows their locations.
The downloads contain the wheel, source archive and SHA256SUMS.
After downloading all three files into one directory, verify them on Linux
with sha256sum --check SHA256SUMS.
Release validation targets Linux with Python 3.12: backend and route tests,
installed commands and resources, and user-storage preservation. It does not
establish macOS or Windows support or validate every live benchmark/provider
combination. These are Python packages with a local web UI.
v0.1.0rc1 · First release candidate
Bring anything. Run it. Prove it. Nullius brings AI evaluations onto your
machines and opens each score into the evidence behind it.
Install the latest version with uv and Python 3.12+:
uv tool install nullius-eval nullius serve
Open http://127.0.0.1:5057.
Full Changelog: All commits in v0.1.0rc1
The package is nullius-eval; the application and command remain nullius.
To select this specific candidate, use uv tool install nullius-eval==0.1.0rc1.
Alternatively, download the wheel attached to this release and run
uv tool install ./nullius_eval-0.1.0rc1-py3-none-any.whl.
To replace an existing Nullius tool installation, add --force to the install
command. Settings, catalogue additions and runs
are stored outside the installed package; nullius paths shows their locations.
Reading existing results does not need a model key. Running benchmarks needs the
appropriate provider credentials and benchmark prerequisites; Docker is needed
for Harbor and the standard's docker backend.
.whl is the installable Python package, including the local web UI.nullius_eval-0.1.0rc1.tar.gz is the source distribution for building the package.SHA256SUMS records the checksums of both files. After downloading all threesha256sum --check SHA256SUMS.These are Python packages, not standalone desktop installers. Release
validation targets Linux with Python 3.12 and checks the backend, web routes,
installed commands and package resources. It also checks that replacing an
installation preserves user storage. This does not establish macOS or Windows
support, or validate every benchmark against live model providers.
Report installation problems, missing assets, incorrect results or confusing
workflows through the issue tracker.
Include the version, operating system, reproduction steps and relevant errors,
with credentials and personal data removed. GitHub downloads and issue access
follow the repository's access permissions.