Speculative-Decoding
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
- 117
- Python
- MIT
- 24 mo ago
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
Awesome-LLM-Prompt-Optimization: a curated list of advanced prompt optimization and tuning methods in Large Language Models
Nadir is a Python package designed to dynamically choose the best llm for your prompt by balancing complexity and cost and response time.
29 free GEO/AI-search Agent Skills plus an installable CLI for audits, citability, AI crawlers, prompt packs, owner maps, Claude Code and Codex. MIT.
Open-source, model-portable coding agent built for cost-efficient, verifiable software work—with model routing, progressive context, compact tool output, explicit task state, and estimated savings.
⚡ Cut LLM inference costs 80% with Programmatic Tool Calling. Instead of N tool call round-trips, generate JavaScript to orchestrate tools in Vercel Sandbox. Supports Anthropic, OpenAI, 100+ models via AI Gateway. Novel MCP Bridge for external service integration.
aiseo-audit analyzes a URL and reports how effectively its content can be ingested, understood, and reused by generative AI engines like ChatGPT, Claude, Gemini, and Perplexity.
Context Excavator solves a real problem: when you paste large codebases into ChatGPT, the AI gets lost in noise. You're feeding it 500 lines of code when only 20 lines actually matter. The LLM can't tell the difference, so it gives irrelevant answers. This project extracts only the code that matters, automatically..
Comprehensive performance analysis of DeepSeek V3 quantization levels (FP16, Q8_0, Q4_0) on 16GB GPU environments.
GPU-accelerated IBM Granite Code model optimization achieving 3-5x performance improvement. Complete benchmarking suite with real-time monitoring and visualization.
Fast MLX port of ZeroEntropy zerank-2 cross-encoder reranker. 10x faster than PyTorch MPS on Apple Silicon. bf16, validated.