sglang

Open repos and models their makers tagged sglang, most starred first. The same for everyone.

AI on my own machineRepo

inference

Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.

  • 9.7k
  • Python
  • Apache-2.0
  • 1 h ago
xorbitsaiTake it
AI on my own machineRepo

gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

  • 6.1k
  • Python
  • Apache-2.0
  • 1 d ago
gpustackTake it
AI on my own machineRepo

auto-round

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率,量化感知训练的精度

  • 1.7k
  • Python
  • Apache-2.0
  • 22 h ago
intelTake it
AI on my own machineRepo

ome

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

  • 528
  • Go
  • Apache-2.0
  • 1 d ago
ome-projectsTake it
AI on my own machineRepo

smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

  • 733
  • Rust
  • Apache-2.0
  • 58 min ago
smg-projectTake it
AI on my own machineRepo

llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

  • 314
  • Go
  • Apache-2.0
  • 3 mo ago
InftyAITake it
Run my own servicesRepo

alzeo

Alzeo is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

  • 270
  • Python
  • MIT
  • 1 h ago
hasso5703Take it
AI on my own machineRepo

gpt_server

gpt_server是一个用于生产级部署LLMs、Embedding、Reranker、ASR、TTS、文生图、图片编辑和文生视频的开源框架。

  • 254
  • Python
  • Apache-2.0
  • 7 mo ago
shell-nlpTake it
AI on my own machineRepo

sglang-rtxpro6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

  • 205
  • Python
  • Apache-2.0
  • 11 h ago
jpezzulliTake it
AI on my own machineRepo

deepseek-local-guide-ru

🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки

  • 197
  • Shell
  • 5 wk ago
justxorTake it
AI on my own machineRepo

spark-doctor

Local diagnostic CLI for NVIDIA DGX Spark (GB10). Detects power caps, UMA pressure, thermal risk, CUDA 13/SM_121 wheel mismatches, Docker issues, and validates vLLM/Ollama/llama.cpp/SGLang recipes.

  • 166
  • Python
  • MIT
  • 2 d ago
joeynycTake it
AI on my own machineRepo

GPU-Perf-Playground

GPU 性能与 AI Infra 学习项目:CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构

  • 126
  • Python
  • Apache-2.0
  • 2 mo ago
yanght27Take it
AI on my own machineRepo

ReadyLLM

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

  • 101
  • Python
  • MIT
  • 2 wk ago
htobtyTake it
HomelabRepo

spark-scope

Read-only dashboard and rack panel for NVIDIA DGX Spark-class nodes, other NVIDIA GPU Linux machines and Apple Silicon Macs running vLLM, SGLang, TensorFold, llama.cpp, bloomery, NVIDIA Flare, Strata or oMLX

  • 70
  • JavaScript
  • MIT
  • 4 wk ago
juliankang4Take it
AI on my own machineRepo

comfyui-openai-api

Call LLMs and VLMs from ComfyUI with any OpenAI compatible chat completions API: local servers (llama.cpp, vLLM, SGLang, Ollama, LM Studio, tabbyAPI, Unsloth Studio…) or cloud providers. Text and image inputs, multi-turn history with memory loops, and model unloading for local servers.

  • 57
  • Python
  • MIT
  • 15 h ago
hekmonTake it
Search GitHub and Hugging Face for sglang

Get , open it once and sign in. Then come back here and press the button again.

sglang: open source repos and models on Take My Repo