unlocking_state_tracking
Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without added training or inference costs.
- 22
- Python
- 21 mo ago
Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without added training or inference costs.
Text generation model, used with transformers.
Gated access
DeltaProduct is a new linear recurrent neural network architecture that uses products of generalized Householder matrices as state-transition matrices enabling better expressivity, state-tracking, and language modeling performance than DeltaNet.
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
Text generation model, used with transformers.
Text generation model, used with transformers.
Text generation model, used with transformers.
Gated access
Text generation model, used with transformers.
Gated access
Text generation model, used with transformers.
Text generation model, used with transformers.
The Delta Rule Lab — error-correcting memory in linear attention (DeltaNet / Gated DeltaNet). Real in-browser autodiff, live MQAR training, gradient-checked.