ECC
The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
- 309k
- JavaScript
- MIT
- 3 d ago

Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability.
Repo api7/aisix, read from GitHub 29 min ago. api7 has not joined Take My Repo.
api7/aisix is a public repo on GitHub by api7. Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability. It declares the Apache-2.0 license on GitHub. It is written mainly in Rust, has 231 stars and last changed on 2026-11-20. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-22.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
One OpenAI-compatible API in front of every model. Route, govern, secure, cache, and observe all your LLM and AI-agent traffic from a single control point — shipped as one static binary with low per-request overhead. Run it in your infrastructure for free, forever.
Built by the original creators of Apache APISIX.
Start free · Documentation · Quickstart · AISIX Enterprise and AISIX Cloud · Roadmap
AISIX OSS is the open-source product built around the Rust-native AISIX gateway. The gateway puts a single, OpenAI-compatible API in front of every LLM provider — OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, DeepSeek, and any OpenAI-compatible endpoint. It gives platform teams one place to route, govern, secure, and observe LLM traffic, with first-class SSE streaming and low gateway overhead.
It runs as a single static binary — low cold-start, lock-free config reads, and hot
configuration reloads with no restarts: declare resources in one resources.yaml and
reload on SIGHUP, or point the gateway at etcd for a multi-replica cluster. Run the
gateway in your infrastructure, or connect it to AISIX Enterprise or
AISIX Cloud
for centralized management with team governance, budgets, audit, and a dashboard.
AISIX OSS (this repo) is the open-source product. It runs without a control plane using declarative configuration or etcd. The same gateway binary serves as the data plane in the commercial products: customers host the control plane with AISIX Enterprise, while API7 hosts it with AISIX Cloud. In all three products, the gateway runs in your environment and calls providers directly; live AI traffic does not pass through either control plane. The proxy API is identical throughout. Talk to us about AISIX Enterprise or AISIX Cloud →
One container. No control plane, no database, no configuration store — the gateway reads
every dynamic resource from one declarative resources.yaml.
# config.yaml
resources_file: /etc/aisix/resources.yaml
proxy:
addr: "0.0.0.0:3000"
admin:
enabled: false # a declarative gateway needs no admin listener
observability:
metrics:
prometheus:
enabled: true
addr: "0.0.0.0:9090"
# resources.yaml — every entry is an id plus the resource's document
_format_version: "2"
provider_keys:
- id: 0b6f3c1e-2f0a-4c5e-9d55-7d1a8b0c2e11
display_name: openai-main
provider: openai
api_key: ${OPENAI_API_KEY} # interpolated from the environment
models:
- id: 6efd592d-a73b-573d-8e2c-e9e103692f92
display_name: my-model
provider: openai
model_name: gpt-4o-mini
provider_key_id: 0b6f3c1e-2f0a-4c5e-9d55-7d1a8b0c2e11
api_keys:
- id: 4a1c9e2b-7d3f-4b6a-8c5e-1f2a3b4c5d6e
display_name: local-dev
key_hash: ${CALLER_API_KEY_SHA256} # sha256 hex of the caller key; the plaintext is never stored
allowed_model_ids: ["6efd592d-a73b-573d-8e2c-e9e103692f92"]
export OPENAI_API_KEY="YOUR_PROVIDER_KEY"
export CALLER_API_KEY="YOUR_CALLER_KEY"
export CALLER_API_KEY_SHA256="$(printf %s "$CALLER_API_KEY" | sha256sum | cut -d' ' -f1)"
docker run -d --name aisix \
-v "$(pwd)/config.yaml:/etc/aisix/config.yaml:ro" \
-v "$(pwd)/resources.yaml:/etc/aisix/resources.yaml:ro" \
-e OPENAI_API_KEY -e CALLER_API_KEY_SHA256 \
-p 3000:3000 -p 127.0.0.1:9090:9090 \
ghcr.io/api7/aisix:latest # proxy → :3000, metrics + status → :9090
# ^ the metrics/status listener is unauthenticated;
# keep it on loopback or a private network
Then call the gateway exactly like OpenAI:
curl http://localhost:3000/v1/chat/completions \
-H "Authorization: Bearer $CALLER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'
Edit resources.yaml and send SIGHUP (docker kill -s HUP aisix) to apply changes with
no restart. The file is loaded exactly as etcd is: a resource that does not load is dropped
and reported on /status/config while the rest serve, and a file that cannot be read at all
keeps the last configuration serving. Check a file before booting with
aisix validate --resources resources.yaml.
Full walkthrough: the
Gateway Quickstart ·
every field: the resources file reference
(still being updated for format "2"; each entry's fields are the resource's document, see
schemas/resources/).
For a multi-replica cluster, point the gateway at etcd instead — resources_file and
etcd are mutually exclusive.
base_url and switch models without changing code.Covered by 183 end-to-end scenario files (496 cases) that run against real gateway processes.
:3000) — chat/completions, completions, responses,
embeddings, rerank, images/{generations,edits}, audio/{speech,transcriptions,translations},
videos (submit → poll → fetch), files, batches, fine_tuning/jobs, realtime,
GET /v1/models, plus a root-level /passthrough/:provider/* escape hatch. Native SSE streaming,
tool/function calling, JSON mode, vision/multimodal input, and reasoning-content support.POST /v1/messages as a first-class route, working against
any configured upstream: requests and responses (including streaming) are translated
both ways when a model points at a non-Anthropic provider.round_robin
(smooth weighted round-robin), consistent_hash (session affinity keyed by header /
cookie / API key / client IP), failover, plus metric-based least_cost,
least_latency, and least_busy. Per-target priority tiers (active/backup pools),
retry budgets, cooldowns, tag-conditional targets, and per-attempt timeouts.api_key / model / team /
member / team_member). Counters are per-process by default, or shared across replicas
with the Redis backend.422 content_filter;
monitor mode records what would have happened without blocking./mcp with gateway-held
credentials, per-server tool namespaces, and per-caller access. It serves every
Streamable HTTP revision from 2025-03-26 through stateless 2026-07-28 without
downstream sessions. Upstreams use initialize by default or server/discover with
protocol_version: "2026-07-28". CI runs the official MCP suite's applicable tools-only
protocol scenarios. Also exposes a REST API as MCP tools from its OpenAPI description./a2a/:agent, serving each
agent's card with URLs rewritten to the gateway, over JSON-RPC 2.0./metrics, structured per-request access logs, usage
events, OTLP/GenAI span export (Langfuse, Honeycomb, Grafana Cloud, or any OTLP receiver),
plus dedicated Datadog and Aliyun SLS log exporters and object-storage (S3/GCS/Azure Blob)
telemetry.resources.yaml carries all ten resource collections
(provider keys, models, caller keys, guardrails, MCP servers, A2A agents, cache policies,
observability exporters, rate-limit policies, OIDC providers) as the same documents the
gateway reads from etcd, loaded by the same code. aisix validate checks a file offline;
SIGHUP reloads it./livez and /readyz on the proxy listener; /status/config,
/status/ready, /status/models, and Prometheus /metrics on a dedicated metrics
listener (:9090). The admin listener (:3001) additionally serves a read-only
resource surface, OpenAPI 3 with a Scalar UI, and a playground. Resources are managed
declaratively — through the resources_file (reloaded on SIGHUP) or direct etcd
writes — not through the admin listener; its former write endpoints were removed.AISIX dispatches through five native adapter families — distinct wire-protocol bridges, not one generic relabel. Whatever the upstream protocol, the client-facing API stays OpenAI-shaped.
| Adapter family | Reaches | Wire shape · auth |
|---|---|---|
openai |
OpenAI + any OpenAI-compatible vendor — DeepSeek, Groq, Mistral, Together, Fireworks, Perplexity, vLLM, Ollama, or self-hosted OpenAI-compatible endpoints | OpenAI chat completions · Bearer |
anthropic |
Anthropic Claude | Anthropic Messages · x-api-key |
bedrock |
AWS Bedrock — Anthropic, Meta Llama, Mistral, Cohere, Amazon Titan/Nova, AI21 | Bedrock Converse + /invoke · SigV4 |
vertex |
Google Vertex AI (Gemini) | Vertex :generateContent · OAuth2 |
azure-openai |
Azure OpenAI | Azure deployments · api-key / Entra ID |
Plus specialized handling for vendor quirks (e.g. DeepSeek reasoning content) and dedicated rerank / embeddings vendors (Cohere, Jina). Details in adapter protocol families.
Same gateway binary, same proxy API — in every form the gateway runs in your environment. AISIX Enterprise adds a customer-hosted commercial control plane. AISIX Cloud adds an API7-hosted commercial control plane.
![]() Overview — traffic, latency, error rate & spend at a glance ![]() Models — one alias per upstream: OpenAI, Anthropic, Bedrock, DeepSeek… ![]() Guardrails — pre-input & post-output policies, block on violation |
![]() Playground — test any model with live token & cost metering ![]() Observability — fan out traces & logs to OTLP, Datadog, object storage ![]() Budgets — hard-stop spend caps with warn-only tiers |
The AISIX Enterprise and AISIX Cloud dashboard — overview metrics, multi-provider models, guardrails, budgets, and observability exporters across all your gateways.
▶ Try the live dashboard demo — aisix-demo.api7.ai
| AISIX OSS (this repo) | AISIX Enterprise | AISIX Cloud | |
|---|---|---|---|
| License | Apache-2.0 | Commercial | Commercial |
| Control plane | None | Customer-hosted | API7-hosted |
| Configuration | Declarative resources.yaml, or etcd for a cluster |
Dashboard + AISIX Cloud Admin API | Dashboard + AISIX Cloud Admin API |
| Tenancy | Single instance / namespace | Organization-scoped environments, teams, and members | Organization-scoped environments, teams, and members |
| Provider keys | ${VAR} references in resources.yaml; plaintext credentials in etcd |
Encrypted at rest; operator-triggered in-place rotation | Encrypted at rest; operator-triggered in-place rotation |
| Budgets | — (rate and token limits only) | Per key, provider, environment, organization, and team | Per key, provider, environment, organization, and team |
| RBAC and audit | Local admin key; no organization layer | Organization roles, invitations, and audit history | Organization roles, invitations, and audit history |
| Operations | You operate the gateway | You operate the control plane and gateways | API7 operates the control plane; you operate gateways |
→ Want centralized governance, budgets, and a dashboard? Talk to API7 about AISIX Enterprise or AISIX Cloud, or book a demo.
A single Cargo workspace; the aisix-server crate builds one binary named aisix that
wires the crates together.
crates/
├── aisix-core Config, snapshot, resource model, errors
├── aisix-etcd Config providers (etcd, resources.yaml) + watch supervisor
├── aisix-gateway Hub & bridge, SSE parser, provider trait
├── aisix-proxy /v1/*, /mcp, /a2a handlers, routing, middleware
├── aisix-admin Read-only resource surface + playground + OpenAPI
├── aisix-provider-* openai · anthropic · azure-openai · bedrock · vertex
├── aisix-mcp MCP gateway — server registry, tool ACL, transports
├── aisix-a2a A2A agent gateway — agent cards, JSON-RPC bridge
├── aisix-ratelimit fixed-window + token accounting + concurrency (local | redis)
├── aisix-cache memory + redis backends
├── aisix-redis shared Redis connection for cache + rate limits
├── aisix-guardrails pre/post content-policy hooks
├── aisix-obs tracing, metrics, access log, exporters
└── aisix-server the `aisix` binary — bootstrap + CLI
Highlights on the roadmap; tracked live in issues:
Shipped since this list was last written: the MCP gateway, the A2A agent gateway, OIDC/JWT inbound auth, Redis-backed distributed rate limiting, and the Lakera, Presidio, PII, and OpenAI Moderation guardrails — see Features above.
Prerequisites: the Rust toolchain pinned in rust-toolchain.toml. Docker is only needed
for the tests that exercise etcd, Redis, or provider emulators.
cargo check --workspace
cargo fmt --check
cargo clippy --workspace -- -D warnings
cargo test --workspace
# Coverage (matches the CI gate)
cargo llvm-cov --workspace --lcov --output-path lcov.info
# Run locally against a resources.yaml (no etcd needed). Copy the Quickstart's two files
# and change resources_file to the local path, e.g. resources_file: ./resources.yaml
cargo run -p aisix-server --bin aisix -- --config config.local.yaml
# Check a resources file without starting a listener
cargo run -p aisix-server --bin aisix -- validate --resources resources.yaml
If AISIX is useful to you, a ⭐ helps other engineers find it.
api7/aisix is a public repo on GitHub by api7. Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability.
It declares the Apache-2.0 license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-11-20. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 900+ contributors
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Adaptive Radix Trees implemented in Lua / LuaJIT
High-performance match IP address for Nginx + Lua
This page is also available as structured data, with the same sources and dates.
/api/v1/resources/gh/api7/aisixTake My Repo has not tested this on your machine. Requirements come from the project's own documentation.