vllm

Open repos and models their makers tagged vllm, most starred first. The same for everyone.

Voice and audioRepo

FunASR

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

  • 21k
  • Python
  • MIT
  • 1 d ago
modelscopeTake it
AI on my own machineRepo

HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

  • 2.7k
  • Python
  • Apache-2.0
  • 8 d ago
syv-aiTake it
AI on my own machineRepo

taOS

Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).

  • 611
  • Python
  • AGPL-3.0
  • 17 h ago
jaylfcTake it
AI on my own machineRepo

OmniSpace

本地优先的全栈 AI 创作工作站:AI对话 · AI漫画 · 漫剧创作 · 写作台 · 知识学习五大模块;vLLM / llama.cpp / Transformers 多引擎本地推理,显存智能调度、舱壁隔离自愈、仅供学习使用,欢迎交流指正。(最新进度(除插件系统)受到协议限制,暂时无法提交到仓库)

  • 536
  • Python
  • 8 wk ago
Yzw202011Take it
AI on my own machineRepo

MACaQ

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

  • 357
  • C++
  • Apache-2.0
  • 1 d ago
huawei-cslTake it
AI on my own machineRepo

flama

The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.

  • 307
  • Python
  • Apache-2.0
  • 2 d ago
vorticoTake it
Search GitHub and Hugging Face for vllm

Send to which agent?

goes to the agent you pick.

Get , open it once and sign in. Then come back here and press the button again.