fast-inference
flux-fp8-api
Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.
- 284
- Python
- Apache-2.0
- 2 y ago
Speculative-Decoding
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
- 117
- Python
- MIT
- 24 mo ago
dsh-jev-interceptor
⚡ Millisecond System-1 judgement for every tool call in DeepSeek Harness — Jev-powered risk classification & evidence-gated auto-approval. Fail-closed by construction. dsh 生态第一个 System-1 决策插件
- 34
- TypeScript
- MIT
- 3 d ago
Sparsifiner
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
- 15
- Python
- MIT
- 3 y ago
Local-Jev
⚡ 100% offline, zero-cost drop-in alternative to TypeSafe Jev System One decision models in Rust (<18ms on CPU)
- 11
- Rust
- MIT
- 2 d ago
zerank-2-mlx
Fast MLX port of ZeroEntropy zerank-2 cross-encoder reranker. 10x faster than PyTorch MPS on Apple Silicon. bf16, validated.
- 0
- Python
- 8 mo ago



