local-inference

Open repos and models their makers tagged local-inference, most starred first. The same for everyone.

GitHubRepo

Mano-P

Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook — no data leaves your device.Mano-P 是一个开源 GUI-VLA 项目,支持在 Mac mini/MacBook 上或通过算力棒本地运行推理,实现纯视觉驱动的跨平台 GUI 自动化操作。数据完全本地处理,支持复杂多步骤任务规划与执行。

  • 2.9k
  • Apache-2.0
  • 5 mo ago
Run my own servicesRepo

ollaya

Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.

  • 1.5k
  • Rust
  • Apache-2.0
  • 1 d ago
ollaya-devTake it
AI on my own machineRepo

kronk

Go With Your Own Intelligence! Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp, and stable-diffusion.cpp directly integrated into your Go applications. Kronk provides a high-level API and production ready model server.

  • 871
  • Go
  • Apache-2.0
  • 23 h ago
ardanlabsTake it
AI on my own machineRepo

MACaQ

SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.

  • 378
  • C++
  • Apache-2.0
  • 3 d ago
huawei-cslTake it
AI on my own machineRepo

Promethevs

PROMETHEVS: run the full DeepSeek-V4.1-Flash-1017 (510 GB) on a gaming PC with 64 GB RAM and a 12 GB GPU. PROMETHEVS engine (compress-then-correct), OpenAI-compatible local server, report and logs.

  • 192
  • CC-BY-4.0
  • 11 d ago
Magnvs7Take it
AI agentsRepo

localharness

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harness (detailed event steams, logs, and performance data) or quickly replace closed source one (featuring out of the box plugins for web search, image gen, memory, auto research and more in dev).

  • 75
  • Python
  • MIT
  • 1 d ago
ahwurmTake it
Voice and audioRepo

neurobrix

Run real AI models on the GPU you already own: language, speech, image and video models from one engine and one container format, with or without PyTorch. NVIDIA GPUs, Mac GPUs through Metal, CPU; other GPUs experimental. Apache 2.0.

  • 65
  • Python
  • Apache-2.0
  • 21 h ago
NeuroBrixTake it
AI on my own machineRepo

lm-supply

.NET library for on-demand local AI model inference — zero bundled models, lazy loading, hardware-aware GPU/CPU selection, 10 task types including embeddings, generation, vision, and audio.

  • 13
  • C#
  • MIT
  • 1 d ago
iyulabTake it
AI on my own machineRepo

quant-lab

Quantization pipelines for local inference: recipes, exact module coverage, and side-by-side evaluation (KLD / top-1 / PPL, decode + TTFT) against both full-precision and vendor checkpoints.

  • 0
  • Python
  • Apache-2.0
  • 4 wk ago
ozdokTake it
Search GitHub and Hugging Face for local-inference

Get , open it once and sign in. Then come back here and press the button again.