SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
- 1.3k
- Python
- MIT
- 24 h ago
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
A scalable, agentic-first, and HuggingFace-native RL framework for research (9k lines).
Best practices & guides on how to write distributed pytorch training code
From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.
A nice ML framework based on Jax/Flax. Continuation of https://github.com/AshishKumar4/FlaxDiff
NemotronH MoE LoRA in NeMo Automodel: routed-expert weights silently not loaded. Root cause, fix script, tests.