all-reduce

Open repos and models their makers tagged all-reduce, most starred first. The same for everyone.

GitHubRepo

distributed_systems_for_ml

A 6-week masterclass in Distributed Deep Learning Infrastructure from scratch. Covers the $\alpha-\beta$ communication model, custom Ring All-Reduce primitives, Parameter Servers (Hogwild!), 1F1B Pipeline Parallelism, ZeRO/FSDP memory sharding, gradient compression with error feedback, and fault-tolerant elastic training simulators.

  • 0
  • Jupyter Notebook
  • MIT
  • 6 mo ago
HAYDARKILICTake it
Search GitHub and Hugging Face for all-reduce

Get , open it once and sign in. Then come back here and press the button again.