Causality
- 471
- Jupyter Notebook
- MIT
The maintainer has not written a description at the source.
Repo DataForScience/LLMs, read from GitHub 1 min ago. DataForScience has not joined Take My Repo.
DataForScience/LLMs is a public repo on GitHub by DataForScience. It declares the MIT license on GitHub. It is written mainly in Jupyter Notebook, has 68 stars and last changed on 2026-11-03. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-18.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Educational notebooks exploring production-grade agentic systems and LLMs pipelines from first principles. Learn the core patterns behind tools like Claude Code, Cursor's agent mode, and autonomous research assistants by implementing them yourself.
These notebooks teach you to build the scaffolding that transforms an LLM from a text generator into a truly useful everyday tool.
| Notebooks | Blog post | Content |
|---|---|---|
| 01 - Basic Agentic Harness.ipynb | Building a Basic Agentic Harness | Start here to understand the fundamentals |
| 02 - Small Language Model.ipynb | Build a (small) language model by counting | Build an n-gram language model from scratch and generate text |
| 03 - Advanced Agentic Harness.ipynb | Building an Advanced Agentic Harness | Production-grade patterns: DAG orchestration, memory, verification, and multi-agent roles |
| None | Self hosting LLMs with Llama.cpp | Run LLMs on your own hardware: install llama.cpp, serve models over an OpenAI-compatible API, explore GGUF files, and choose the right quantization |
| 04 - Evaluating Agentic Harnesses.ipynb | Evaluating Your Agentic Harness | Eval suites, cost and latency measurement, failure modes, and three measured upgrades |
| 05 - MCP Server.ipynb | An MCP Server from Scratch | Build an MCP server from scratch: OpenAlex data pipeline, SQLite + FTS5, raw JSON-RPC over stdio, and a from-scratch client |
| None | LLMs Are the New Wikipedia | Every old criticism is new again: why the reliability debate around LLMs mirrors Wikipedia's early years, and why quantitative evaluation is what settles it |
| 06 - Local Inference with vLLM.ipynb | Local LLM Inference at Scale with vLLM | Run an open-weight model with vLLM: choosing a model from memory bandwidth, continuous batching, prefix caching, and structured outputs over thousands of abstracts |
| 07 - Jev 101.ipynb | Jev 101: A new class of model | Meet Jev, TypeSafe's System 1 model: typed Noul, Choice, and Score questions through the TypeSafe SDK, calibrated probabilities instead of text, and many questions billed as one call |
01 - Basic Agentic Harness.ipynb) to understand the core concepts02 - Small Language Model.ipynb) to see how language models work from the ground up03 - Advanced Agentic Harness.ipynb) to upgrade the harness with production-grade patterns04 - Evaluating Agentic Harnesses.ipynb) to measure whether any of it actually works05 - MCP Server.ipynb) to hand real data to an agent through a from-scratch MCP server06 - Local Inference with vLLM.ipynb) to run an open-weight model on your own GPU and process thousands of documents with it07 - Jev 101.ipynb) to meet a model that makes typed decisions instead of generating textNotebooks 3 and 4 share a module, d4sci_harness.py — notebook 3 builds it
step by step, notebook 4 imports it. See The harness as a module below.
Notebook 5 likewise keeps its executable pieces in mcp_server/ — see
The MCP server as scripts below. Notebook 6 reads the OpenAlex
database that notebook 5 builds, and needs a Linux machine with an NVIDIA GPU — see
Hardware for notebook 6 below.
Build a minimal but complete harness from scratch — the foundation of any autonomous agent system.
Core concepts:
What you'll build: A single-agent harness that solves multi-step research tasks by repeatedly composing context, asking the LLM what to do next, executing tool calls, and updating state until the goal is met.
Build a complete language model from first principles — counting, not neural networks — to understand what every LLM is really doing under the hood.
Core concepts:
P(next word | previous words)What you'll build: A 4-gram model trained on Wikipedia that generates text from a prompt, with a temperature switch that mirrors the same decoding knob exposed by modern LLMs.
Upgrade every component toward production-grade systems like Claude Code, Devin, or modern research agents.
Advanced topics:
What you'll build: A three-city comparison agent whose plan contains nine independent fetches — executed in parallel under a concurrency cap — plus a final aggregation step and verifiable output, with trace plots for per-step latency, budget pressure, and tokens by role.
One good demo proves the harness can work. An eval suite proves it usually works — and prices the times it doesn't.
Core concepts:
failed_execute vs. failed_verify), regression baselines, and which verification tier produced each verdictClosing three gaps, each shipped with its own mini-benchmark:
all-MiniLM-L6-v2 vs. Jaccard retrieval, benchmarked over a labeled query set rather than swapped on faithFetcherAgent / WriterAgent split routed by capability, because millisecond lookups and multi-second LLM calls do not deserve the same concurrency and retry policyWhat you'll build: A four-task eval suite run against the harness module, producing a metrics table and four diagnostic plots built entirely from the structured trace the orchestrator already emits — no extra instrumentation.
Build a complete MCP server from scratch — raw JSON-RPC over stdio, no SDK — that gives an agent queryable access to a real bibliographic database.
Core concepts:
select= field trimming, caching the raw JSON separately from the database_meta, result envelopes, cache hints, and error classification, all made visible2026-07-28, the current spec version, which made the core stateless: server/discover replaces the initialize handshake, and every request carries its own version and capabilities in _metaWhat you'll build: A five-tool, one-resource MCP server over an OpenAlex subset — list_tables, describe_table, query with handle-based pagination, fetch_page, and BM25 search_works — plus the launch configuration to plug it into a real MCP host.
Stop treating the model as a remote service: run an open-weight LLM yourself with vLLM, the engine most self-hosted deployments run on, and measure what it does.
Core concepts:
Qwen3.5-35B-A3B-FP8) beats dense models on a 128 GB DGX Spark.LLM(...) does at startup (weight loading, memory profiling, torch.compile, CUDA graphs) and where the memory goesvllm serve and an OpenAI-compatible APIWhat you'll build: A structured extraction over 2,000 "scaling laws" abstracts sampled from notebook 5's OpenAlex database, turned into an analysis of which fields use the term, whether the model agrees with OpenAlex's own topic labels, and how often a "scaling law" is actually a power law.
Not every LLM call needs to generate text. Jev, TypeSafe's "System 1" model, answers typed questions about a piece of input with calibrated probabilities, and never writes a word.
Core concepts:
state (string, JSON object, or array), a mapping of named questions, and a model ID, sent through the typed TypeSafe Python SDKNoul — a yes/no question that returns one probability, which is its own confidenceChoice — one label from up to 255, with a probability for every option, so you see what the model almost pickedScore — a position on an ordered scale, returned as a probability-weighted score you round yourselfWhat you'll build: A small banking-support triage, with team routing, urgency, and refund detection, answered by Jev in a single call.
Everything notebook 3 builds step by step — typed tools, the plan DAG, the parallel executor, multi-tier memory, the verification hierarchy, multi-dimensional budgets, structured tracing, and the Orchestrator that composes them — also lives in d4sci_harness.py as an importable module. Notebook 3 constructs it; notebook 4 imports it, which is how you would consume it in a real project:
import d4sci_harness as dh from d4sci_harness import TOOLS, MemoryStore, Orchestrator llm = dh.set_provider("anthropic") # or "mock" for offline runs store = MemoryStore() orch = Orchestrator(provider=llm, tools=TOOLS, memory=store) result = await orch.run("Compare Paris and Tokyo.", ["paris", "tokyo"]) print(result.status, result.budget.tokens_used, result.verdict.tier)
| Subsystem | Key names |
|---|---|
| LLM providers | LLMProvider, AnthropicProvider, MockProvider, set_provider |
| Typed tools | TypedTool, TOOLS, CityArgs, AggregateArgs |
| Plan as a DAG | PlanDAG, PlanNode, NodeStatus, PlanDAG.plot |
| Parallel execution | execute_dag, MAX_CONCURRENT, MAX_NODE_RETRIES |
| Memory | MemoryStore, WorkingMemory, build_context |
| Verification | Verdict, ReportCheck, deterministic_check_report, llm_judge_report, verify_report |
| Agent roles | PlannerAgent, WorkerAgent, CriticAgent |
| Budget + recovery | BudgetMulti, ErrorClass, classify_error, retry_with_backoff |
| Tracing | TraceEvent, Tracer |
| Composition root | Orchestrator, RunResult |
Two things worth knowing before you extend it:
MockProvider is fully offline and rule-based; AnthropicProvider uses Claude. The code path is identical either way, which is what makes the eval suite runnable in CI.Orchestrator.run() accepts a check callable that replaces the deterministic tier wholesale, so you can verify something other than "does this text mention these terms" without touching the escalation logic above it.Notebook 5 explains every design decision in prose, but the executable pieces live in mcp_server/ so they can run outside the notebook — from a shell, a Makefile, or an MCP host's launch configuration:
| File | Role |
|---|---|
mcp_openalex_server.py |
The dependency-free MCP stdio server: five tools + one resource over the read-only OpenAlex database |
create_openalex_db.py |
The build pipeline: raw JSONL cache → parse_work → two-pass bulk load → FTS index. Run as python create_openalex_db.py [raw_path] [db_path] |
mcp_openalex_adapter.py |
A dual-era adapter that answers the legacy (pre-2026) initialize handshake itself and bridges older clients to the stateless server without modifying it |
test_mcp_openalex_server.py |
Wire-level tests that spawn the server as a subprocess and exercise it exactly as a host would |
Everything on the server's critical path is standard library only (json, sqlite3, sys), so it runs as a bare subprocess wherever a host launches it. The server is modern-only: it requires a host that speaks MCP 2026-07-28, the latest protocol revision, and rejects the legacy initialize handshake explicitly — hosts still on an earlier version should launch mcp_openalex_adapter.py instead. To register it with a stdio host, point the host at the server script with absolute paths:
{
"mcpServers": {
"openalex": {
"command": "/ABS/PATH/TO/python3",
"args": ["/ABS/PATH/TO/mcp_server/mcp_openalex_server.py"]
}
}
}The server resolves data/openalex.db relative to its own location, so it is independent of whatever working directory the host chooses.
The repository also ships a project-scoped .mcp.json, read by hosts such as Claude Code, that registers the server through mcp_openalex_adapter.py. Its command points at the author's virtual environment, so change it to the Python interpreter in your own .venv before using it.
LLMs/
├── 01 - Basic Agentic Harness.ipynb # Notebook 1: Fundamentals
├── 02 - Small Language Model.ipynb # Notebook 2: n-gram language model
├── 03 - Advanced Agentic Harness.ipynb # Notebook 3: Production-grade patterns
├── 04 - Evaluating Agentic Harnesses.ipynb # Notebook 4: Eval suite, cost, failure modes
├── 05 - MCP Server.ipynb # Notebook 5: MCP server from scratch
├── 06 - Local Inference with vLLM.ipynb # Notebook 6: Self-hosted inference with vLLM
├── 07 - Jev 101.ipynb # Notebook 7: Typed decisions with Jev
├── d4sci_harness.py # The harness from notebook 3, importable
├── mcp_server/ # Notebook 5's executable pieces
│ ├── mcp_openalex_server.py # The from-scratch MCP stdio server
│ ├── create_openalex_db.py # OpenAlex → SQLite build pipeline
│ ├── mcp_openalex_adapter.py # Dual-era protocol adapter
│ └── test_mcp_openalex_server.py # Wire-level server tests
├── .mcp.json # Project MCP config that launches the adapter
├── data/
│ ├── D4Sci_logo_ball.png
│ ├── D4Sci_logo_full.png # Logos and assets
│ ├── bgoncalves.png # Author photo
│ ├── openalex_raw.jsonl.gz # Raw OpenAlex subset data
│ └── openalex.db # OpenAlex SQLite database
├── outputs/ # Everything the notebooks write, one folder per notebook
│ ├── small_language_model/cache/ # Notebook 2's cached 4-gram model
│ ├── advanced_harness/ # Notebook 3's plan DAG figures
│ ├── evaluating_harnesses/ # Notebook 4's re-plan DAG figure
│ └── vllm/ # Notebook 6's extractions.jsonl
│ └── cache/ # Notebook 6's cached generations and measurements
├── d4sci.mplstyle # Custom matplotlib style
├── pyproject.toml # Dependency manifest (for `uv sync`)
├── uv.lock # Lock file for reproducible builds
└── LICENSE # MIT License
uv)uv (if needed):curl -LsSf https://astral.sh/uv/install.sh | shgit clone https://github.com/DataForScience/LLMs.git cd LLMs uv venv source .venv/bin/activate # Windows: .venv\Scripts\activate uv sync
The project requires Python 3.13 or newer; uv venv picks a matching interpreter from pyproject.toml and downloads one if needed.
Notebook 1 is the only notebook that needs an Anthropic API key: it calls Claude through AnthropicProvider and has no offline fallback. Notebook 7 needs a TypeSafe key (see below). Notebooks 3 and 4 include a rule-based mock provider that is smart enough to drive the demos, notebooks 2 and 5 don't call an LLM at all, and notebook 6 runs its model locally (see Hardware for notebook 6).
Notebooks 3 and 4 ship with BACKEND = "anthropic", so to run them offline set:
BACKEND = "mock"
For notebook 1, and for notebooks 3 and 4 on the real backend, export your Anthropic API key before launching Jupyter:
export ANTHROPIC_API_KEY=sk-ant-...Notebook 7 calls Jev live through the TypeSafe SDK, so it needs a TypeSafe API key:
export TYPESAFE_API_KEY=...Notebook 5 needs an OPENALEX_API_KEY only for a fresh data fetch — the cached raw
data lets the rest of the notebook run without credentials:
export OPENALEX_API_KEY=...Mock mode is deterministic, which makes it the right choice for CI and for following along. Real-backend runs are not — plans vary between runs, so expect the eval suite's pass rate to move around. That variability is the point of measuring it.
Keys aside, the first run of several notebooks downloads data: notebook 2 fetches WikiText-103
from Hugging Face, notebooks 3 and 4 fetch the all-MiniLM-L6-v2 embedding model, and notebook 6
fetches its model weights.
Notebook 6 loads a real model into GPU memory, so it needs a Linux machine with an NVIDIA GPU. uv sync installs vLLM and its GGUF plugin only on Linux
(both dependencies carry a sys_platform == 'linux' marker), so on macOS the rest of the
environment installs normally and notebook 6 is the one to skip. On Linux, the lock file pins
vLLM 0.31.0 on PyTorch 2.13 with CUDA 13 wheels, so your NVIDIA driver must support CUDA 13.
MEMORY_BANDWIDTH_GBPS and DEVICE_MEMORY_GB in the configuration cell to your device's figures and adjust GPU_MEMORY_UTILIZATION to the memory you can spare; the model-choice arithmetic in Section 1 then tells you whether the default model still fits comfortably.Qwen/Qwen3.5-35B-A3B-FP8 is a public, Apache-2.0 checkpoint, so no Hugging Face token is needed, but the first run downloads about 37 GB of weights, and the first engine start spends several minutes compiling the model and capturing CUDA graphs.data/openalex.db, the database built in notebook 5.outputs/vllm/cache/, keyed on the vLLM version, the engine arguments, and the inputs. With the same vLLM version and settings, a re-run reloads those results instead of regenerating them (the engine itself still starts). The repository ships the cache from the original DGX Spark run, made with the vLLM version in uv.lock, so a fresh clone reloads those results too. Set REFRESH_CACHE = True to measure everything again, for instance on different hardware.jupyter notebook
Reach out at info@data4sci.com or open an issue if something isn't working.
|
Bruno GonçalvesData For Science, Inc.
Web: www.data4sci.com |
DataForScience/LLMs is a public repo on GitHub by DataForScience. Its maker has not written a description.
It declares the MIT license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-11-03. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
This page is also available as structured data, with the same sources and dates.
/api/v1/resources/gh/DataForScience/LLMsTake My Repo has not tested this on your machine. Requirements come from the project's own documentation.