vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
- 922
- Python
- Apache-2.0
- 13 d ago
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
RACS (Remote Agent Context Store): prefix-cache management for production agents. Stability-aware prompt planning, provider-faithful cache directives for Anthropic, OpenAI, Gemini, Bedrock and more, TTL keep-warm scheduling, prefix-drift detection, and hit-ratio and savings analytics. Zero dependencies, TypeScript, edge-ready.
ComfyUI Qwen-Image 2.1 multi-GPU inference: Ulysses sequence parallelism for 1/2/4/8 GPUs, T2I, image editing, INT8/FP8, and distributed prefix cache.
Cost governance for Massive Intelligence (IM) agent orchestration: hard per-request, per-task, and per-day USD budgets with depletion events, pre-flight cost forecasting, prefix-cache break-even planning across sixteen provider profiles, and cheapest-first cascade routing. Zero dependencies, node-free core, deterministic, TypeScript-first.
LLM API caching middleware that combines exact-match, semantic, and upstream prefix-cache accounting for OpenAI-compatible providers.