inference-acceleration

Open repos and models their makers tagged inference-acceleration, most starred first. The same for everyone.

AI on my own machineRepo

SageAttention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

  • 4k
  • Cuda
  • Apache-2.0
  • 10 mo ago
thu-mlTake it
AI on my own machineRepo

gravestone

Hardware probe and best-of-N sampling harness in C. Ranks GGUF quantisations against measured VRAM and RAM, drives Ollama or llama.cpp over OpenAI-format HTTP on localhost, and gates every candidate behind a static check and a test run.

  • 1
  • C
  • 2 mo ago
ahmedwaqas92Take it
Search GitHub and Hugging Face for inference-acceleration

Get , open it once and sign in. Then come back here and press the button again.