Shipped: RAGnarok-AI — Evaluate RAG systems locally, without external APIs

RAGnarok-AI is an open-source framework for evaluating Retrieval-Augmented Generation (RAG) systems, with a focus on local execution and reproducible results.

It helps developers measure whether a RAG pipeline retrieves relevant information and generates answers grounded in its context.

The project includes:

• Retrieval and generation evaluation metrics
• LLM-as-a-judge evaluation
• Local model support, including offline evaluation workflows
• CLI tools for running evaluations
• Quality gates that can be integrated into CI/CD pipelines
• Adapters for common RAG frameworks and vector databases

The goal is to make RAG evaluation accessible without sending private datasets or documents to external APIs.

I'm actively developing the framework and working on more reliable evaluation methods, including local LLM judges and reproducible benchmarks.

GitHub: github.com/2501Pr0ject/RAGnarok-AI