llm-d
State of the art inference performance with modern accelerators on Kubernetes
- 5.3k
- Python
- Apache-2.0
- 1 d ago
State of the art inference performance with modern accelerators on Kubernetes
High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.
High Performan Ai Model Web Server. Mainly support computer vision model. Quickly establish your own ai-model server. https://github.com/MaybeShewill-CV/mortred_model_server