speculative-decoding

Open repos and models their makers tagged speculative-decoding, most starred first. The same for everyone.

AI agentsRepo

Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

  • 4.2k
  • Python
  • Apache-2.0
  • 2 h ago
raullenchaiTake it
AI on my own machineRepo

HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

  • 2.7k
  • Python
  • Apache-2.0
  • 8 d ago
syv-aiTake it
AI on my own machineRepo

MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

  • 2.7k
  • Python
  • Apache-2.0
  • 11 d ago
youssofalTake it
AI on my own machineRepo

PolyStrata

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

  • 1.2k
  • C++
  • MIT
  • 7 h ago
simplexai-labsTake it
AI on my own machineRepo

mlx-dspark

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

  • 741
  • Python
  • MIT
  • 10 d ago
ARahim3Take it
AI on my own machineRepo

janas

37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

  • 735
  • C
  • GPL-3.0
  • 3 h ago
prabanta-devTake it
AI on my own machineRepo

llamAmpere

llama.cpp fork for significantly improved performance on Ampere generation Nvidia cards (especially RTX 3090 / 3090 Ti): example: 110+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx. 120+ TPS, 3300+ prefill for QFN 3.8.

  • 393
  • C++
  • MIT
  • 5 h ago
JakeATXTake it
Search GitHub and Hugging Face for speculative-decoding

Send to which agent?

goes to the agent you pick.

Get , open it once and sign in. Then come back here and press the button again.