aicomp
- 23
- Python
- 2 d ago
ai-computing
Systems for heterogeneous AI infrastructure: AI-RM resource management, Onggi artifact storage and delivery, and the Optimus Prime engine.
Built from ai-computing's public GitHub profile. Not on Take My Repo yet, so there are no posts or updates here.
ai-rm L1: accelerator discovery and AcceleratorProfile publishing for GPUs and NPUs (FuriosaAI RNGD, Rebellions ATOM), plus a vLLM-compatible API
Content-addressed store for model weights and accelerator-compiled artifacts on Kubernetes + llm-d
Layered KV-cache transport between heterogeneous accelerators (GPU, RNGD, ATOM, host): Rust core, C ABI, Python bindings
PD-disaggregated serving across Furiosa RNGD (prefill) and Rebellions ATOM (decode) as llm-d pods
Feasibility study: switching NPU compiled graphs (sequence length, batch) while the weights stay loaded — ATOM and RNGD
K8s + llm-d heterogeneous cluster (A100, RNGD, ATOM, CPU): manifests, EPP, model store, monitoring
Completion-fenced same-host heterogeneous NPU KV transport (RNGD -> ATOM), llm-d stage integration
Heterogeneous AI resource integration for Kubernetes + llm-d, with Optimus Prime and FASOP as engine consolidation targets.
LoRA / full fine-tuning on inference-only RNGD NPU (npu-lora), compared against A100
RNGD MoE expert offloading (hot experts on NPU HBM, cold experts on host CPU)
Terminal UI for observing and operating heterogeneous Kubernetes + llm-d clusters
Contracts that connect ai-rm components: accelerators, memory tiers, KV transport, placement, metrics (protocol v0.1)
ai-rm load generation and measurement tools (trace replay, synthetic load) for llm-d endpoints