When assessing a Retrieval Augmented Generation (RAG) application using Databricks Agent Evaluation with various Large Language Model (LLM) judges, a team needs to determine which evaluation metric necessitates a pre-existing, labeled 'correct' answer for its scoring. Which of the following judges ABSOLUTELY REQUIRES ground truth data for evaluation?
Answer & explanation
Correct answer: D
WHY D is correct: The correctness judge is an LLM judge that compares the generated answer with a labeled reference: the expected_response (or expected_facts) in the evaluation set. Without that labeled reference it has nothing to judge against, so it cannot run without ground truth.
WHY NOT A: Groundedness (faithfulness) measures consistency between the generated response and the retrieved document chunks — both are available at inference time in a RAG system. No pre-labeled ground truth is needed; the judge only needs the response and its source context.
WHY NOT B: The safety judge looks only at the response text and decides whether it contains harmful or offensive content. It needs no labeled reference answer.
WHY NOT C: Relevance to query evaluates whether the response addresses the user's question — it only requires the question and the response, both available at inference time. No pre-labeled expected answer is needed to judge topical alignment.