inference-engine

Open repos and models their makers tagged inference-engine, most starred first. The same for everyone.

AI on my own machineRepo

MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

  • 2.7k
  • Python
  • Apache-2.0
  • 11 d ago
youssofalTake it
AI on my own machineRepo

PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

  • 1.2k
  • C++
  • MIT
  • 7 h ago
simplexai-labsTake it
AI on my own machineRepo

mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

  • 741
  • Python
  • MIT
  • 10 d ago
ARahim3Take it
AI on my own machineRepo

janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

  • 735
  • C
  • GPL-3.0
  • 3 h ago
prabanta-devTake it
AI on my own machineRepo

slotstream

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

  • 673
  • Swift
  • MIT
  • 13 h ago
carloslfuTake it
Search GitHub and Hugging Face for inference-engine

Send to which agent?

goes to the agent you pick.

Get , open it once and sign in. Then come back here and press the button again.

inference-engine: open source repos and models on Take My Repo