tensorrt-llm

Open repos and models their makers tagged tensorrt-llm, most starred first. The same for everyone.

GitHubRepo

grps

Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.

  • 172
  • C++
  • Apache-2.0
  • 19 mo ago
AI on my own machineRepo

grps_trtllm

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

  • 160
  • Python
  • Apache-2.0
  • 12 mo ago
AI on my own machineRepo

GPU-Perf-Playground

GPU 性能与 AI Infra 学习项目:CUDA/Triton 算子、NCU/NSYS、vLLM/SGLang/TRT-LLM/ms-swift、PyTorch/DeepSpeed/ms-swift 训练、并行架构

  • 126
  • Python
  • Apache-2.0
  • 2 mo ago
yanght27Take it
Search GitHub and Hugging Face for tensorrt-llm

Get , open it once and sign in. Then come back here and press the button again.