offloading

Open repos and models their makers tagged offloading, most starred first. The same for everyone.

AI on my own machineRepo

Xolotl

Xolotl: run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 110 GB RAM): ~33 tok/s agent, ~41 fast, 512K context. Plus Inkling-2-Mini (Thinking Machines, own engine + MTP drafter), GLM-5.3-Flash, Qwen3.8, LFM3-Omni. MoE experts tiered across VRAM, RAM and NVMe (Xolotl Direct). OpenAI + Anthropic API.

  • 548
  • C++
  • MIT
  • 11 h ago
AI on my own machineRepo

Abyss

Run 50-280 GB MoE LLMs (DeepSeek V4/V4.1 Flash, GLM-5.3-Flash, Qwen3.8-Flash-Next, Ornith) on one consumer GPU: experts spread over VRAM, RAM and NVMe SSD. Browser chat + OpenAI/Anthropic APIs. CUDA, ROCm, DGX Spark.

  • 41
  • C++
  • MIT
  • 11 h ago
zephel01Take it
Search GitHub and Hugging Face for offloading

Get , open it once and sign in. Then come back here and press the button again.

offloading: open source repos and models on Take My Repo