gpt4all
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
- 77k
- C++
- MIT
- 18 mo ago
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Fast and Accurate Code Search for Agents. Uses 99% fewer tokens than grep+read
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Long-term memory runtime for AI agents — plain Markdown as the source of truth, local ranked retrieval, and an independent sleep-time Manage layer. Claude Code and Codex share one store. No API key.
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.
| Name | Stars | License | Language | Last change | In common | Compare |
|---|---|---|---|---|---|---|
| FasterDecoding/REST | 221 | Apache-2.0 | C | 9 mo ago | ||
| nomic-ai/gpt4all | 77k | MIT | C++ | 18 mo ago | llm-inference | Compare |
| VectifyAI/PageIndex | 41k | MIT | Python | 18 h ago | retrieval | Compare |
| raullenchai/Rapid-MLX | 4.2k | Apache-2.0 | Python | 42 min ago | llm-inference, speculative-decoding | Compare |
| ray-project/ray | 44k | Apache-2.0 | Python | 2 h ago | llm-inference | Compare |
| MinishLab/semble | 6.4k | MIT | Python | 13 d ago | retrieval | Compare |
| Luce-Org/lucebox | 3k | Apache-2.0 | C++ | 1 d ago | speculative-decoding | Compare |
| gitleaks/gitleaks | 31k | MIT | Go | 3 wk ago | llm-inference | Compare |
| VectifyAI/OpenKB | 5k | Apache-2.0 | Python | 4 mo ago | retrieval | Compare |
| syv-ai/HyperQwen | 2.8k | Apache-2.0 | Python | 11 d ago | llm-inference, speculative-decoding | Compare |
| liguodongiot/llm-action | 25k | Apache-2.0 | HTML | 5 wk ago | llm-inference | Compare |
| tigerless-labs/agent-memory | 4.3k | MIT | Python | 9 d ago | retrieval | Compare |
| youssofal/MTPLX | 2.7k | Apache-2.0 | Python | 2 wk ago | llm-inference, speculative-decoding | Compare |