# Ragas
> Evaluation framework for your AI Application
Ragas is an open-source evaluation framework for LLM applications including RAG pipelines,
AI agents, and workflows. It provides objective metrics for evaluation, test data generation
capabilities, and integrations with popular LLM frameworks like LangChain and LlamaIndex.
## Getting Started
- [🚀 Get Started](https://docs.ragas.io/en/stable/getstarted/index.md)
- [Evaluate a simple LLM application](https://docs.ragas.io/en/stable/getstarted/evals/index.md)
- [Run your first experiment](https://docs.ragas.io/en/stable/getstarted/experiments_quickstart/index.md)
- [Installation](https://docs.ragas.io/en/stable/getstarted/install/index.md)
- [Quick Start](https://docs.ragas.io/en/stable/getstarted/quickstart/index.md)
- [Evaluate a simple RAG system](https://docs.ragas.io/en/stable/getstarted/rag_eval/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/getstarted/rag_testset_generation/index.md)
## Tutorials
- [Tutorials](https://docs.ragas.io/en/stable/tutorials/index.md)
- [Evaluate an AI Agent](https://docs.ragas.io/en/stable/tutorials/agent/index.md)
- [Evaluate a prompt](https://docs.ragas.io/en/stable/tutorials/prompt/index.md)
- [Evaluate a simple RAG system](https://docs.ragas.io/en/stable/tutorials/rag/index.md)
- [Evaluate an AI Workflow](https://docs.ragas.io/en/stable/tutorials/workflow/index.md)
## Core Concepts
- [📚 Core Concepts](https://docs.ragas.io/en/stable/concepts/index.md)
- [Datasets](https://docs.ragas.io/en/stable/concepts/datasets/index.md)
- [Experimentation](https://docs.ragas.io/en/stable/concepts/experimentation/index.md)
- [Components Guide](https://docs.ragas.io/en/stable/concepts/components/index.md)
- [Evaluation Dataset](https://docs.ragas.io/en/stable/concepts/components/eval_dataset/index.md)
- [Evaluation Sample](https://docs.ragas.io/en/stable/concepts/components/eval_sample/index.md)
- [Prompt](https://docs.ragas.io/en/stable/concepts/components/prompt/index.md)
- [Utilizing User Feedback](https://docs.ragas.io/en/stable/concepts/feedback/index.md)
- [Metrics](https://docs.ragas.io/en/stable/concepts/metrics/index.md)
- [List of available metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/index.md)
- [Agentic or Tool use](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/agents/index.md)
- [Answer correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_correctness/index.md)
- [Response Relevancy](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_relevance/index.md)
- [Aspect Critique](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/aspect_critic/index.md)
- [Context Entities Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_entities_recall/index.md)
- [Context Precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/index.md)
- [Context Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/index.md)
- [Factual Correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/factual_correctness/index.md)
- [Faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/index.md)
- [General Purpose Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/general_purpose/index.md)
- [Multi modal faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_faithfulness/index.md)
- [Multi modal relevance](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_relevance/index.md)
- [Noise Sensitivity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/noise_sensitivity/index.md)
- [Nvidia Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/nvidia_metrics/index.md)
- [Rubric-Based Evaluation](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/rubrics_based/index.md)
- [Semantic Similarity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/semantic_similarity/index.md)
- [SQL](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/sql/index.md)
- [Summarization](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/summarization_score/index.md)
- [Traditional NLP Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/traditional/index.md)
- [Overview](https://docs.ragas.io/en/stable/concepts/metrics/overview/index.md)
- [Testset Generation](https://docs.ragas.io/en/stable/concepts/test_data_generation/index.md)
- [Testset Generation for Agents or Tool use cases](https://docs.ragas.io/en/stable/concepts/test_data_generation/agents/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/concepts/test_data_generation/rag/index.md)
## Metrics
- [Overview](https://docs.ragas.io/en/stable/concepts/metrics/overview/index.md)
- [List of available metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/index.md)
- [Agentic or Tool use](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/agents/index.md)
- [Answer correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_correctness/index.md)
- [Response Relevancy](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/answer_relevance/index.md)
- [Aspect Critique](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/aspect_critic/index.md)
- [Context Entities Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_entities_recall/index.md)
- [Context Precision](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_precision/index.md)
- [Context Recall](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/context_recall/index.md)
- [Factual Correctness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/factual_correctness/index.md)
- [Faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/faithfulness/index.md)
- [General Purpose Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/general_purpose/index.md)
- [Multi modal faithfulness](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_faithfulness/index.md)
- [Multi modal relevance](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/multi_modal_relevance/index.md)
- [Noise Sensitivity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/noise_sensitivity/index.md)
- [Nvidia Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/nvidia_metrics/index.md)
- [Rubric-Based Evaluation](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/rubrics_based/index.md)
- [Semantic Similarity](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/semantic_similarity/index.md)
- [SQL](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/sql/index.md)
- [Summarization](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/summarization_score/index.md)
- [Traditional NLP Metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/traditional/index.md)
## Test Data Generation
- [Testset Generation](https://docs.ragas.io/en/stable/concepts/test_data_generation/index.md)
- [Testset Generation for Agents or Tool use cases](https://docs.ragas.io/en/stable/concepts/test_data_generation/agents/index.md)
- [Testset Generation for RAG](https://docs.ragas.io/en/stable/concepts/test_data_generation/rag/index.md)
## Customization Guides
- [Customizations](https://docs.ragas.io/en/stable/howtos/customizations/index.md)
- [Caching](https://docs.ragas.io/en/stable/howtos/customizations/_caching/index.md)
- [Cancelling Tasks](https://docs.ragas.io/en/stable/howtos/customizations/cancellation/index.md)
- [Customise models](https://docs.ragas.io/en/stable/howtos/customizations/customize_models/index.md)
- [Run Config](https://docs.ragas.io/en/stable/howtos/customizations/run_config/index.md)
- [Understand Cost and Usage of Operations](https://docs.ragas.io/en/stable/howtos/customizations/metrics/_cost/index.md)
- [Adapting Metrics to Target Language](https://docs.ragas.io/en/stable/howtos/customizations/metrics/metrics_language_adaptation/index.md)
- [Modify Prompts](https://docs.ragas.io/en/stable/howtos/customizations/metrics/modifying-prompts-metrics/index.md)
- [Tracing and logging evaluations with Observability tools](https://docs.ragas.io/en/stable/howtos/customizations/metrics/tracing/index.md)
- [DSPy Optimizer](https://docs.ragas.io/en/stable/howtos/customizations/optimizers/index.md)
- [Customizing Test Data Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/index.md)
- [Non-English Testset Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_language_adaptation/index.md)
- [Persona Generation](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_persona_generator/index.md)
- [Custom Single-hop Query](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_testgen-custom-single-hop/index.md)
- [Custom Multi-hop Query](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/_testgen-customisation/index.md)
- [Using Pre-chunked Data](https://docs.ragas.io/en/stable/howtos/customizations/testgenerator/prechunked_data/index.md)
## Application Guides
- [Applications](https://docs.ragas.io/en/stable/howtos/applications/index.md)
- [Cost Analysis](https://docs.ragas.io/en/stable/howtos/applications/_cost/index.md)
- [Adding to your CI pipeline with Pytest](https://docs.ragas.io/en/stable/howtos/applications/add_to_ci/index.md)
- [Align an LLM as a Judge](https://docs.ragas.io/en/stable/howtos/applications/align-llm-as-judge/index.md)
- [Evaluate a New LLM](https://docs.ragas.io/en/stable/howtos/applications/benchmark_llm/index.md)
- [Compare Embeddings for retriever](https://docs.ragas.io/en/stable/howtos/applications/compare_embeddings/index.md)
- [Compare LLMs using Ragas Evaluations](https://docs.ragas.io/en/stable/howtos/applications/compare_llms/index.md)
- [Evaluate and Improve a RAG App](https://docs.ragas.io/en/stable/howtos/applications/evaluate-and-improve-rag/index.md)
- [Evaluating Multi-turn Conversations](https://docs.ragas.io/en/stable/howtos/applications/evaluating_multi_turn_conversations/index.md)
- [Iterate and Improve Prompts](https://docs.ragas.io/en/stable/howtos/applications/iterate_prompt/index.md)
- [Systematic Prompt Optimization](https://docs.ragas.io/en/stable/howtos/applications/prompt_optimization/index.md)
- [Single-hop Query Testset](https://docs.ragas.io/en/stable/howtos/applications/singlehop_testset_gen/index.md)
- [Evaluate a Text-to-SQL Agent](https://docs.ragas.io/en/stable/howtos/applications/text2sql/index.md)
- [Aligning LLM Evaluators with Human Judgment](https://docs.ragas.io/en/stable/howtos/applications/vertexai_alignment/index.md)
- [Compare models provided by VertexAI on RAG-based Q&A task using Ragas metrics](https://docs.ragas.io/en/stable/howtos/applications/vertexai_model_comparision/index.md)
- [Evaluations with Vertex AI models](https://docs.ragas.io/en/stable/howtos/applications/vertexai_x_ragas/index.md)
## CLI
- [Ragas CLI](https://docs.ragas.io/en/stable/howtos/cli/index.md)
- [Agent Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/agent_evals/index.md)
- [LLM Benchmarking Quickstart](https://docs.ragas.io/en/stable/howtos/cli/benchmark_llm/index.md)
- [Improve RAG](https://docs.ragas.io/en/stable/howtos/cli/improve_rag/index.md)
- [Judge Alignment Quickstart](https://docs.ragas.io/en/stable/howtos/cli/judge_alignment/index.md)
- [LlamaIndex Agent Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/llamaIndex_agent_evals/index.md)
- [Prompt Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/prompt_evals/index.md)
- [RAG Evaluation](https://docs.ragas.io/en/stable/howtos/cli/rag_eval/index.md)
- [Text-to-SQL Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/text2sql/index.md)
- [Workflow Evaluation Quickstart](https://docs.ragas.io/en/stable/howtos/cli/workflow_eval/index.md)
## Integrations
- [Integrations](https://docs.ragas.io/en/stable/howtos/integrations/index.md)
- [AG-UI Integration](https://docs.ragas.io/en/stable/howtos/integrations/_ag_ui/index.md)
- [Arize](https://docs.ragas.io/en/stable/howtos/integrations/_arize/index.md)
- [Athina AI](https://docs.ragas.io/en/stable/howtos/integrations/_athina/index.md)
- [Haystack Integration](https://docs.ragas.io/en/stable/howtos/integrations/_haystack/index.md)
- [Helicone](https://docs.ragas.io/en/stable/howtos/integrations/_helicone/index.md)
- [Langchain](https://docs.ragas.io/en/stable/howtos/integrations/_langchain/index.md)
- [Langfuse](https://docs.ragas.io/en/stable/howtos/integrations/_langfuse/index.md)
- [LangGraph](https://docs.ragas.io/en/stable/howtos/integrations/_langgraph_agent_evaluation/index.md)
- [Langsmith](https://docs.ragas.io/en/stable/howtos/integrations/_langsmith/index.md)
- [LlamaIndex](https://docs.ragas.io/en/stable/howtos/integrations/_llamaindex/index.md)
- [Openlayer](https://docs.ragas.io/en/stable/howtos/integrations/_openlayer/index.md)
- [Comet Opik](https://docs.ragas.io/en/stable/howtos/integrations/_opik/index.md)
- [Tonic Validate](https://docs.ragas.io/en/stable/howtos/integrations/_tonic-validate/index.md)
- [Zeno](https://docs.ragas.io/en/stable/howtos/integrations/_zeno/index.md)
- [AG-UI](https://docs.ragas.io/en/stable/howtos/integrations/ag_ui/index.md)
- [Amazon Bedrock](https://docs.ragas.io/en/stable/howtos/integrations/amazon_bedrock/index.md)
- [Google Gemini](https://docs.ragas.io/en/stable/howtos/integrations/gemini/index.md)
- [Griptape](https://docs.ragas.io/en/stable/howtos/integrations/griptape/index.md)
- [Haystack](https://docs.ragas.io/en/stable/howtos/integrations/haystack/index.md)
- [LangChain](https://docs.ragas.io/en/stable/howtos/integrations/langchain/index.md)
- [LangSmith](https://docs.ragas.io/en/stable/howtos/integrations/langsmith/index.md)
- [LlamaStack](https://docs.ragas.io/en/stable/howtos/integrations/llama_stack/index.md)
- [LlamaIndex Agents](https://docs.ragas.io/en/stable/howtos/integrations/llamaindex_agents/index.md)
- [OCI Gen AI](https://docs.ragas.io/en/stable/howtos/integrations/oci_genai/index.md)
- [R2R](https://docs.ragas.io/en/stable/howtos/integrations/r2r/index.md)
- [Swarm](https://docs.ragas.io/en/stable/howtos/integrations/swarm_agent_evaluation/index.md)
## API Reference
- [API References](https://docs.ragas.io/en/stable/references/index.md)
- [aevaluate()](https://docs.ragas.io/en/stable/references/aevaluate/index.md)
- [Cache](https://docs.ragas.io/en/stable/references/cache/index.md)
- [Embeddings](https://docs.ragas.io/en/stable/references/embeddings/index.md)
- [evaluate()](https://docs.ragas.io/en/stable/references/evaluate/index.md)
- [Schemas](https://docs.ragas.io/en/stable/references/evaluation_schema/index.md)
- [Executor](https://docs.ragas.io/en/stable/references/executor/index.md)
- [Generation](https://docs.ragas.io/en/stable/references/generate/index.md)
- [Graph](https://docs.ragas.io/en/stable/references/graph/index.md)
- [Integrations](https://docs.ragas.io/en/stable/references/integrations/index.md)
- [LLMs](https://docs.ragas.io/en/stable/references/llms/index.md)
- [Metrics](https://docs.ragas.io/en/stable/references/metrics/index.md)
- [Optimizers](https://docs.ragas.io/en/stable/references/optimizers/index.md)
- [Prompt](https://docs.ragas.io/en/stable/references/prompt/index.md)
- [RunConfig](https://docs.ragas.io/en/stable/references/run_config/index.md)
- [Synthesizers](https://docs.ragas.io/en/stable/references/synthesizers/index.md)
- [Schemas](https://docs.ragas.io/en/stable/references/testset_schema/index.md)
- [Tokenizers](https://docs.ragas.io/en/stable/references/tokenizers/index.md)
- [Transforms](https://docs.ragas.io/en/stable/references/transforms/index.md)
Measured from the saved file
Linked document
UTF-8 text size
15.8 KB16197 bytes
Token count
4210o200k_base encoding
Markdown links
157133 distinct URLs
Fenced code blocks
0196 lines in the file
Size measures the captured UTF-8 text, not compressed transfer size. Tokens are counted with a named tokenizer; other models can count differently. Neither measure establishes context compatibility or answer quality.
Project H1
Ragas
Opening blockquote
Evaluation framework for your AI Application
Context architecture
How the content is divided
10 H2 sections organize 157 parsed Markdown links. 157 links point to .md or .markdown paths; 0 link occurrences use another hostname. Linked destinations have not been recursively validated.
H2 section
Links
UTF-8 bytes
Tokens
Getting StartedLine 10
7
639
169
TutorialsLine 20
5
410
113
Core ConceptsLine 28
33
3479
866
MetricsLine 64
21
2396
590
Test Data GenerationLine 88
3
335
83
Customization GuidesLine 94
16
1825
441
Application GuidesLine 113
16
1848
482
CLILine 132
10
947
271
IntegrationsLine 145
27
2361
703
API ReferenceLine 175
19
1454
399
Link counts exclude parsed fenced and inline code and resolve relative destinations against the final endpoint. Section sizes cover the text below each H2 until the next H2.
Embedded material
Code, installation and API examples
Fenced blocks
0 closed
Declared languages
No fenced blocks detected
Package-install commands
0 detected in fenced or inline code
Authentication examples
0 fenced lines match API-key, Authorization, Bearer, OAuth or access-token patterns
HTTP request examples
0 fenced lines match request-method patterns
Authentication links
0 links have authentication-related labels
Embedded example text and links to further documentation are counted separately. A code block does not establish that a sample runs or that an authentication flow succeeds.
Publisher prioritization
The Optional section
No H2 section named “Optional” was detected in this file.
The community proposal uses Optional by convention for secondary links that can be skipped for a shorter context. This label does not establish that everything outside it is critical, or that the material inside it is deprecated.
Machine-readable routes
MCP references and configuration text
No MCP or Model Context Protocol term was detected in the captured file. 0 links have MCP-related labels or paths. 0 fenced lines match mcpServers, mcp_servers or an mcp add command.
These are text and link observations. No MCP server connection, authentication or tool call was tested.
A verified text response establishes a companion at the checked URL. It does not prove the bundle contains all product documentation, that generation is automated, or that it fits an assistant’s context window.
Change evidence
What can be said about freshness
This capture establishes a baseline. Update frequency cannot be determined from one observation.
File changes can reveal documentation edits or navigation changes. They do not by themselves measure product release velocity, accuracy, agent adoption or AI visibility.
Counts use a deterministic parser, not a spec-compliance grade. Unusual Markdown and indented code can fall outside its recognized patterns. See the llms.txt proposal and tokenizer implementation.