infinity-scheduler
CPU + GPU scheduler optimized for modern-day desktop interactivity
- 66
- Shell
- GPL-2.0
- 8 wk ago
CPU + GPU scheduler optimized for modern-day desktop interactivity
Manage heterogeneous accelerators, vGPU slicing, shared KV cache, model delivery, and inference workloads.
Self-hosted LLM inference on Kubernetes: agentgateway + Gateway API Inference Extension + llm-d EPP, vLLM (T4) and llama.cpp, KEDA queue-depth autoscaling, Kueue/DRA GPU scheduling, Prometheus/Grafana, and an emulated BGP/EVPN leaf-spine AI fabric with RoCE v2 design.