llm-d

Open repos and models their makers tagged llm-d, most starred first. The same for everyone.

AI on my own machineRepo

llm-d-lab

Accelerate reproducible inference experiments for large language models with LLM-D! This lab automates the setup of a complete evaluation environment on OpenShift/OKD: GPU worker pools, core operators, observability, traffic control, and ready-to-run example workloads.

  • 6
  • Jupyter Notebook
  • Apache-2.0
  • 3 mo ago
aleskandroTake it
AI on my own machineRepo

goodput-stack

Self-hosted LLM inference on Kubernetes: agentgateway + Gateway API Inference Extension + llm-d EPP, vLLM (T4) and llama.cpp, KEDA queue-depth autoscaling, Kueue/DRA GPU scheduling, Prometheus/Grafana, and an emulated BGP/EVPN leaf-spine AI fabric with RoCE v2 design.

  • 1
  • Python
  • Apache-2.0
  • 2 wk ago
veldanda3415Take it
GitHubRepo

llm-d

一个 Kubernetes 原生的高性能分布式大语言模型推理框架

  • 1
  • 13 mo ago
whailiTake it
Search GitHub and Hugging Face for llm-d

Get , open it once and sign in. Then come back here and press the button again.