modern-llm-notebook
A hands-on course on modern LLM architectures, training, and inference, with step-by-step PyTorch implementations and runnable notebooks.
- 290
- Jupyter Notebook
- 7 wk ago
A hands-on course on modern LLM architectures, training, and inference, with step-by-step PyTorch implementations and runnable notebooks.
Train 1B Recurrent-Depth MoE (OpenMythos) AI models from scratch on consumer GPUs (RTX 4060 8GB to RTX 5060 Ti 16GB) -- solo or pooled with friends over HTTP.
A Transformer encoder built from first principles, implementing the core architecture from mathematical foundations to working PyTorch code, including tokenization, embeddings, positional encoding, self-attention, multi-head attention, LayerNorm, feed-forward networks, training, and evaluation.
A from-scratch implementation of a small language model, built part by part, from a text file to a system capable of generation, classification, instruction following, parameter-efficient adaptation, and reasoning-style preference training.