Alternatives to FasterDecoding/REST

FasterDecoding/REST: REST: Retrieval-Based Speculative Decoding, NAACL 2024 License Apache-2.0. Below, the open repos Take My Repo keeps on the same subjects, most starred first. Stars measure popularity, not quality.

AI agentsRepo

Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

  • 4.2k
  • Python
  • Apache-2.0
  • 42 min ago
raullenchaiTake it
AI on my own machineRepo

HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

  • 2.8k
  • Python
  • Apache-2.0
  • 11 d ago
syv-aiTake it
AI on my own machineRepo

MTPLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

  • 2.7k
  • Python
  • Apache-2.0
  • 2 wk ago
youssofalTake it

Side by side

NameStarsLicenseLanguageLast changeIn commonCompare
FasterDecoding/REST221Apache-2.0C9 mo ago
nomic-ai/gpt4all77kMITC++18 mo agollm-inferenceCompare
VectifyAI/PageIndex41kMITPython18 h agoretrievalCompare
raullenchai/Rapid-MLX4.2kApache-2.0Python42 min agollm-inference, speculative-decodingCompare
ray-project/ray44kApache-2.0Python2 h agollm-inferenceCompare
MinishLab/semble6.4kMITPython13 d agoretrievalCompare
Luce-Org/lucebox3kApache-2.0C++1 d agospeculative-decodingCompare
gitleaks/gitleaks31kMITGo3 wk agollm-inferenceCompare
VectifyAI/OpenKB5kApache-2.0Python4 mo agoretrievalCompare
syv-ai/HyperQwen2.8kApache-2.0Python11 d agollm-inference, speculative-decodingCompare
liguodongiot/llm-action25kApache-2.0HTML5 wk agollm-inferenceCompare
tigerless-labs/agent-memory4.3kMITPython9 d agoretrievalCompare
youssofal/MTPLX2.7kApache-2.0Python2 wk agollm-inference, speculative-decodingCompare

Read from GitHub and Hugging Face. A value a source does not give is shown as unknown, never guessed.

Get , open it once and sign in. Then come back here and press the button again.