Turn Markdown into executable tests.
mdproof runs the commands in your docs, runbooks, and smoke tests, then verifies the result with assertions.
Use one .md file for documentation, operational steps, and CI checks.
Best for README example verification, CLI and API smoke tests, deploy runbooks, and agent-generated workflows.
โ ๏ธ Under active development โ APIs and runbook format may change.
What mdproof does
mdproof executes shell steps from a Markdown file and checks the output with assertions like exit_code, jq, substring matching, regex, and snapshots.
That makes Markdown useful for more than prose:
- Your README examples can become runnable checks.
- Your deploy runbooks can become executable verification steps.
- Your smoke tests can stay readable by humans and writable by agents.
Instead of splitting documentation and tests across different formats, mdproof keeps them in one place.
### Step 1: Create a user
```bash
curl -s -X POST http://localhost:8080/users -d '{"name":"alice"}'
```
Expected:
- jq: .id != null
- jq: .name == "alice"
The test is the documentation. When it fails, the error points to the exact step and expectation that broke.
flowchart LR
A["README or runbook in Markdown"] --> B["mdproof runs each shell step"]
B --> C["Assertions verify output and exit status"]
C --> D["Humans and agents fix code or docs from a precise failure"]
Why teams use it
- Docs stop drifting: examples in
README.md or docs/*.md can be verified in CI.
- Runbooks become safer: the same file an operator reads can be executed during deploy verification.
- Agents fit naturally: agents already write Markdown well, so they can generate or repair tests without learning a test framework API.
- Review stays simple: a Markdown diff is easier to inspect than a custom test harness for many workflow-style checks.
- Failures are traceable: mdproof reports the Markdown file and line that failed, so humans and agents can jump straight to the broken step.
Quick Start
1. Install:
curl -fsSL https://raw.githubusercontent.com/runkids/mdproof/main/install.sh | sh
2. Write a test (api-proof.md):
# API Smoke Test
## Steps
### Step 1: Health check
```bash
curl -sf http://localhost:8080/health
```
Expected:
- exit_code: 0
- jq: .status == "ok"
### Step 2: Create item
```bash
curl -s -X POST http://localhost:8080/items \
-H "Content-Type: application/json" \
-d '{"name":"test"}'
```
Expected:
- jq: .id != null
- jq: .name == "test"
3. Run it:
mdproof sandbox api-proof.md # auto-provisions a container
โ api-proof.md
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Step 1 Health check 52ms
โ Step 2 Create item 18ms
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
2/2 passed 80ms
Tip: Run mdproof --help for the full list of flags and subcommands.
When a check fails, mdproof reports where it came from:
โ source-aware-assert-proof.md
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Step 1 Assertion failure 1ms
FAIL runbooks/fixtures/source-aware-assert-proof.md:13 Step 1: Assertion failure
Assertion runbooks/fixtures/source-aware-assert-proof.md:13 expected output
โโ expected: expected output
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
0/1 passed 1 failed 1ms
Use Cases
| Use Case |
How |
| AI agent test loop |
Agent writes .md โ mdproof runs โ JSON report โ agent fixes code โ re-run. Never leaves Markdown. |
| CLI tool E2E testing |
Build โ run โ assert output. Especially good for Go/Rust single-binary CLIs. |
| API smoke testing |
curl + jq: assertions. No Postman, no SDK. The test IS the docs. |
| Deployment verification |
Post-deploy runbook: health checks, DB migration, service connectivity. Ops can read it, CI can run it. |
| README code verification |
--inline mode ensures code examples in docs never go stale. |
Not a fit for: unit tests (use go test/pytest), browser UI (Playwright), perf benchmarks, or complex programmatic fixtures.
Features
|
For AI Agents
- Markdown is native โ no framework API to learn
- Self-contained โ one file = commands + assertions
- JSON / JUnit XML output โ
--report json or --report junit for programmatic parsing
- Built-in skill โ
skills/SKILL.md teaches your agent the full syntax
- Debuggable โ agent reads step, sees output, fixes it
|
For Humans
- Documentation IS the test โ no context switching
- Readable โ anyone can understand what's being tested
- Lifecycle hooks โ
--build, --setup/--teardown, -step-setup/-step-teardown
- Container-first โ safe by default, sandbox mode
- Persistent sessions โ env vars flow across steps and
--- sub-commands
- Per-runbook isolation โ
--isolation per-runbook for clean $HOME/$TMPDIR
- Step filtering โ
--steps 1,3, --from N, --fail-fast
- Coverage โ
--coverage for CI gating
- Inline testing โ
--inline validates code examples in any .md
- GitHub Actions annotations โ
--report github for PR inline error markers
- Zero dependencies โ pure Go stdlib, single binary
|
Prerequisites
| Requirement |
Needed for |
Install |
| Docker |
mdproof sandbox (container-first execution) |
docs.docker.com/get-docker |
| jq (optional) |
jq: assertions |
brew install jq / apt install jq |
Without Docker you can still run with --strict=false, but sandbox mode is recommended for safety.
Install
macOS / Linux
curl -fsSL https://raw.githubusercontent.com/runkids/mdproof/main/install.sh | sh
Windows (PowerShell)
irm https://raw.githubusercontent.com/runkids/mdproof/main/install.ps1 | iex
Homebrew
brew install runkids/tap/mdproof
Tip: Run mdproof upgrade to update to the latest version. It auto-detects your platform and handles the rest.
From Source
go install github.com/runkids/mdproof/cmd/mdproof@latest
Runbook Format
A runbook is a standard Markdown file with step headings, bash code blocks, and Expected: assertions.
# Deploy Verification
## Steps
### Step 1: Check health
```bash
curl -sf http://localhost:8080/health
```
Expected:
- exit_code: 0
- jq: .status == "ok"
### Step 2: Create resource
```bash
curl -s -X POST http://localhost:8080/items \
-d '{"name":"test"}'
```
Expected:
- jq: .id != null
- Should NOT contain error
Assertions
| Type |
Syntax |
Example |
| Substring |
plain text |
- hello world |
| Negated |
No/Should NOT prefix |
- Should NOT contain error |
| Exit code |
exit_code: N |
- exit_code: 0 |
| Regex |
regex: prefix |
- regex: v\d+\.\d+ |
| jq |
jq: prefix |
- jq: .status == "ok" |
| Snapshot |
snapshot: prefix |
- snapshot: api-response |
No Expected: section โ exit code decides (0 = pass).
Key Concepts
- Persistent session โ all steps share one bash process;
export vars persist across steps and --- sub-commands
- Container-first โ strict mode (default) refuses to run outside containers; use
mdproof sandbox or --strict=false
- Hooks โ
--build (once), --setup / --teardown (per runbook), -step-setup / -step-teardown (per step)
- Directives โ
<!-- runbook: timeout=30s retry=3 depends=2 --> for per-step control
- Sub-commands โ
--- separator splits a code block into independent subshells with shared env
- Isolation โ
--isolation per-runbook gives each runbook a fresh $HOME and $TMPDIR
- Step filtering โ
--steps 1,3,5 or --from N to run a subset; --fail-fast to stop early
- Coverage โ
--coverage reports assertion coverage; --coverage-min N for CI gating
- Inline testing โ
--inline extracts <!-- mdproof:start/end --> blocks from any .md
- Source-aware failures โ failed assertions, command exits, and parser errors point back to Markdown file + line
- Snapshots โ
snapshot: assertions with -u to update
Documentation
|
|
| Writing Runbooks |
Full assertion reference, directives, inline testing, persistent sessions, source tracking |
| CLI Reference |
All flags, subcommands, sandbox mode, usage examples, failure output |
| Advanced Features |
Hooks, configuration, reports, coverage, CI integration, architecture |
| Troubleshooting |
Common errors, strict mode, snapshots, jq debugging |
AI Agent Skill
mdproof ships with skills/SKILL.md โ install it once, and your AI agent knows the full syntax.
# Claude Code (via https://github.com/runkids/skillshare)
skillshare install runkids/mdproof
# Manual: copy skills/SKILL.md to your agent's skill directory
License
MIT