w4a8

Open repos and models their makers tagged w4a8, most starred first. The same for everyone.

AI on my own machineRepo

cider

W8A8/W4A8 inference + optimized SDPA on Apple Silicon — unlocking unused INT8 TensorOps in M5 for 1.2–1.9× faster LLM prefill, plus FlashInfer-inspired GQA decode attention for up to 1.6× SDPA speedup, built as MLX custom primitives.

  • 360
  • Python
  • MIT
  • 6 mo ago
AI on my own machineRepo

ljqinfer

Deploy DeepSeek-V4.1 (dsv4.1) on Ascend 910B / 910B3 (NPU): a hardware-specialized LLM inference engine with W4A8, DSpark speculative decoding, 128K+ context and 8,700+ tok/s prefill. Also supports NVIDIA A100/A800. 昇腾910B 部署 DeepSeek-V4.1-Flash。

  • 7
  • C++
  • MIT
  • 6 d ago
lsdefineTake it
Search GitHub and Hugging Face for w4a8

Get , open it once and sign in. Then come back here and press the button again.