🐢 Open-Source Evaluation & Testing library for LLM Agents

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

RAG evaluation without the need for "golden answers"

Red Teaming python-framework for testing chatbots and GenAI systems.

RAG boilerplate with semantic/propositional chunking, hybrid search (BM25 + dense), LLM reranking, query enhancement agents, CrewAI orchestration, Qdrant vector search, Redis/Mongo sessioning, Celery ingestion pipeline, Gradio UI, and an evaluation suite (Hit-Rate, MRR, hybrid configs).

⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or CLI. Privacy-first, async, visual reports.

Open source framework for evaluating AI Agents

Your RAG pipeline is broken and you don't know it. Chunky lets you validate your Markdown and choose the best chunking strategy before indexing.

smallevals — CPU-fast, GPU-blazing fast offline retrieval evaluation for RAG systems with tiny QA models.

Evaluation Framework for LLM applications in Java and Kotlin

This project aims to compare different Retrieval-Augmented Generation (RAG) frameworks in terms of speed and performance.

A framework for systematic evaluation of retrieval strategies and prompt engineering in RAG systems, featuring an interactive chat interface for document analysis.

Learn Retrieval-Augmented Generation (RAG) from Scratch using LLMs from Hugging Face and Langchain or Python

RAG Chatbot for Financial Analysis

Benchmark 7 retrieval strategies on your own docs — naive vector, contextual, QnA pairs, knowledge graph, RAPTOR, PageIndex, and hybrid. Find which KB architecture fits your data.

EntRAG - Enterprise RAG Benchmark

A modular, multi-model AI assistant UI built on .NET 9, featuring RAG, extensible tools, and deep code + database knowledge through semantic search.