llm-d
llm-d-fast-model-actuation
Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping
- 28
- Go
- Apache-2.0
- 3 d ago
llm-d-lab
Accelerate reproducible inference experiments for large language models with LLM-D! This lab automates the setup of a complete evaluation environment on OpenShift/OKD: GPU worker pools, core operators, observability, traffic control, and ready-to-run example workloads.
- 6
- Jupyter Notebook
- Apache-2.0
- 3 mo ago
llm-d-resiliency-manager
Experimental recovery coordination for expert-parallel inference groups in llm-d.
- 3
- Go
- Apache-2.0
- 4 d ago
flow-control-visualizer
Replay and understand llm-d flow-control experiments across client, EPP, and vLLM layers.
- 1
- TypeScript
- MIT
- 2 mo ago
goodput-stack
Self-hosted LLM inference on Kubernetes: agentgateway + Gateway API Inference Extension + llm-d EPP, vLLM (T4) and llama.cpp, KEDA queue-depth autoscaling, Kueue/DRA GPU scheduling, Prometheus/Grafana, and an emulated BGP/EVPN leaf-spine AI fabric with RoCE v2 design.
- 1
- Python
- Apache-2.0
- 2 wk ago
onpar
Prove which open model can replace your closed one — on your traffic, your hardware, your budget. Then move you there without a risky cutover.
- 2
- Python
- Apache-2.0
- 11 d ago
onggi
Content-addressed store for model weights and accelerator-compiled artifacts on Kubernetes + llm-d
- 0
- Go
- Apache-2.0
- 9 d ago
moe-disaggregation-benchmark
Does prefill/decode disaggregation pay for an MoE model? SLO-first goodput benchmark of Qwen3-30B-A3B on 4x H100 and 8x A100 with vLLM, llm-d and NVIDIA Dynamo, with per-request data.
- 0
- Python
- Apache-2.0
- 4 wk ago
llm-d-sc-sensitivity-v3-prior
Text classification model, used with transformers.
- 0
- 18
- apache-2.0
- 3 mo ago
ai-rm
Heterogeneous AI resource integration for Kubernetes + llm-d, with Optimus Prime and FASOP as engine consolidation targets.
- 0
- Python
- BSD-3-Clause
- 4 wk ago
lmd-top
Terminal UI for observing and operating heterogeneous Kubernetes + llm-d clusters
- 0
- Rust
- Apache-2.0
- 5 wk ago
hetero-llm-d
K8s + llm-d heterogeneous cluster (A100, RNGD, ATOM, CPU): manifests, EPP, model store, monitoring
- 0
- Python
- 4 wk ago
PROVENANCE
Verifiable, tenant-isolated LLM inference controls for regulated AI infrastructure.
- 0
- Python
- MIT
- 2 mo ago
Inferentia
Self-hosted LLM inference platform on AWS EKS — vLLM, llm-d, Cosign, ArgoCD, Kyverno, Kagent+MCP
- 0
- HCL
- Apache-2.0
- 2 mo ago
llm-d-sc-sensitivity
Sentence similarity model, used with sentence-transformers.
- 0
- 13
- apache-2.0
- 3 mo ago





