inference-gateway

Open repos and models their makers tagged inference-gateway, most starred first. The same for everyone.

AI on my own machineRepo

aisix

Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability.

  • 227
  • Rust
  • Apache-2.0
  • 3 d ago
api7Take it
AI on my own machineRepo

smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

  • 737
  • Rust
  • Apache-2.0
  • 1 h ago
smg-projectTake it
Developer toolsRepo

cli

A Git-first CLI coding agent that turns ideas, issues, and tasks into real code changes. It can run remotely from your mobile phone or automate your computer.

  • 8
  • Go
  • Apache-2.0
  • 34 min ago
AI on my own machineRepo

gravestone

Hardware probe and best-of-N sampling harness in C. Ranks GGUF quantisations against measured VRAM and RAM, drives Ollama or llama.cpp over OpenAI-format HTTP on localhost, and gates every candidate behind a static check and a test run.

  • 1
  • C
  • 2 mo ago
ahmedwaqas92Take it
Search GitHub and Hugging Face for inference-gateway

Get , open it once and sign in. Then come back here and press the button again.