cs249r_book
Machine Learning Systems: Foundations, Scaling, Agentic AI, and Physical AI (Vols I–IV) • Harvard CS249r | https://mlsysbook.ai
- 30k
- Python
- 8 h ago
Machine Learning Systems: Foundations, Scaling, Agentic AI, and Physical AI (Vols I–IV) • Harvard CS249r | https://mlsysbook.ai
Sub-50mb agentic, voice and vision intelligence for consumer devices.
Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.
[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
This is a list of interesting papers and projects about TinyML.
sanoTTS (सानो = 'small' in Nepali): 294k–2.27M-parameter neural TTS that runs on a $3 chip, in the browser, or in Python with low-latency streaming.
Seeed SenseCraft Model Assistant is an open-source project focused on embedded AI. 🔥🔥🔥
Whistle speech-to-text running fully on an ESP32-S3: hold a button, say a colour in one of 7 languages, the LED does it. No network.
A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.
Oído: open-vocabulary speech recognition on a $5 ESP32-S3, faster than real time. No cloud, no NPU. English + Spanish.
Open-source, fully offline wake word / keyword spotting (KWS) inference engine — ONNX/TFLite <130KB, multi-keyword, multilingual. Train a custom wake word online with auto hard-negative mining; deploy on ESP32 / Raspberry Pi / Android / Web / Windows / macOS / Linux / Home Assistant — runs on any ONNX/TFLite-capable device.
559 K パラメータの日本語 TTS を ESP32-S3 で実時間合成。漢字かな交じり文の形態素解析・アクセント推定まで端末内で走る(M5Stack CoreS3 実機で確認)。推論は依存ゼロの C99、ブラウザ demo あり。arXiv:2608.21378 sanoTTS の日本語 clean-room 再実装。⚠️ コードは MIT ですが、配布モデルの重みは MIT ではありません(LICENSE-MODEL.md。出力に用途制限が伝播します)
TinyML & Edge AI: On-device inference, model quantization, embedded ML, ultra-low-power AI for microcontrollers and IoT devices.
World's First NMS-Free YOLOv26n on ESP32-P4. Features end-to-end Int8 QAT and custom C++ optimizations achieving 30% faster inference than the official ESP-DL YOLOv11n (1.7s vs 2.4s).
Real-time ear biometric recognition on STM32F407 with FreeRTOS - 95%+ accuracy, 120ms latency using PCA/LDA/DCVA on ARM Cortex-M4
Open-source cat collar IMU/BLE behavior data collection prototype with local web logging, OTA/DFU, and baseline modeling.
Unofficial homework and notes for MIT 6.5940: TinyML and Efficient AI Computing (Fall 2026).
Pre-quantized models for edge inference on Cortex-M, ESP32-S3, and Raspberry Pi. Six models — keyword spotting, person detection, hand-gesture recognition, anomaly detection, voice activity detection, and binary defect classifier — each shipped as TF