gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
- 6.1k
- Python
- Apache-2.0
- 2 d ago
gpustack
GPU cluster manager for optimized AI model deployment
Built from gpustack's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
Provides a unified interface to detect GPU resources and manages GPU workloads.
Collection of Dockerfiles to build images for various inference services across different accelerated backends.
Manage heterogeneous accelerators, vGPU slicing, shared KV cache, model delivery, and inference workloads.
Benchmark OpenAI-compatible LLM servers: find throughput peaks and latency SLO capacity, with terminal charts.
An agent harness for bounded, unattended software engineering with sandboxed execution, fenced state, and independent verification.