OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- 13k
- Python
- Apache-2.0
- 5 d ago
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
A simple yet complete guide to MLOps tools and practices - from a conventional way to a modern approach of working with ML projects.
AladdinAI: open-source BYOI (Bring Your Own Infrastructure) AI Agent platform. Connect your own VMs, NVIDIA NIM, OpenAI, Anthropic, Ollama — build and manage AI agents visually.
Serve Llama 3.3 70B (with AWQ quantization) using vLLM and deploy it on BentoCloud.
My repo for the Machine Learning Engineering bootcamp 2022 by DataTalks.Club
Production LLM serving infrastructure using Triton Inference Server, vLLM, and Ray Serve with OpenAI-compatible endpoints. Includes Kubernetes autoscaling configs driven by DCGM GPU metrics and a BentoML packaging path for portable model deployment.
Serving and deploying LLMs with Bento Ecosystem.
build and deploy an audio generation API server using Bark and BentoML ecosystem.
Evaluating several Machine Learning Models in order to predict the price of a phone.