PromptPulse
Automated, production-grade CI/CD evaluation pipeline for Hugging Face chatbots using DeepEval and dynamic JSON test datasets.
- 4
- Python
- MIT
- 6 wk ago
Automated, production-grade CI/CD evaluation pipeline for Hugging Face chatbots using DeepEval and dynamic JSON test datasets.
Learn GenAI and Agentic AI from Zero to Production
Universal Agent Evaluation Framework (UAEF) is a framework-agnostic evaluation system for AI agents. Invoke any agent and score it on multiple metrics spanning tool calling, response quality, safety, performance, and reasoning. Track experiments against baselines, catch regressions automatically, and get LLM-generated insights.
A practical, AI evalution repo covering LLMs, RAG, MCP, skills, agents, multi-agent systems, security, reliability evaluations
Successfully developed a multi-domain RAG pipeline that retrieves and answers questions grounded in research papers on lung cancer detection, NSCLC microbiome classification, and Bhutan land cover, with a full safety evaluation suite covering toxicity, information leakage, and scope adherence using DeepEval.
RAG over my writing + a reproducible DeepEval suite — a working system, not a skills list.
Successfully implemented a retrieval evaluation pipeline that tests how well a RAG system finds relevant call center transcript chunks, using contextual precision and recall as metrics.