semantic-kv-control-plane
A systems research platform for semantic KV-cache orchestration, topology-aware memory placement, distributed prefix reuse, and rack-scale inference memory simulation.
- 1
- Python
- MIT
- 6 mo ago
A systems research platform for semantic KV-cache orchestration, topology-aware memory placement, distributed prefix reuse, and rack-scale inference memory simulation.
Deadline-aware KV-cache scheduling for protecting decode-critical request-state under long-context LLM inference pressure.
Experimental Linux RFC for an HBF/CXL-era AI memory control plane: runtime hints, prefetch, placement, and tiering.
Exploratory AI infrastructure project modeling semantic KV-cache orchestration, memory tiering, and HBM/CXL movement tradeoffs for long-context LLM inference.
Research harness for evaluating query-time bounded elimination of reconstructable KV-cache witnesses in long-context transformer inference workloads. Related provisional filing: IN 202641062451.
Reference Linux kernel interface for workload-aware memory hints in AI systems — /dev/mem_hint maps LLM phases like prefill, decode, agentic execution, and training to memory-policy signaling paths.