A data scientist is comparing two text segmentation approaches for a Retrieval-Augmented Generation (RAG) system handling medical literature. The first approach employs fixed-size recursive character splits (512 tokens with 50-token overlap). The second approach utilizes semantic chunking, aggregating sentences based on their embedding similarity. While the semantic approach yields superior Mean Reciprocal Rank (MRR) during retrieval, it demands considerably more time for initial indexing. When would the semantic chunking approach (Strategy B) be the more suitable option?
Answer & explanation
Correct answer: B
WHY B is correct: Semantic chunking groups sentences by embedding similarity, preserving topical coherence independent of character count. When medical research papers contain dense multi-sentence paragraphs that discuss a single concept, semantic chunking keeps related sentences together — yielding higher MRR because each retrieved chunk fully encapsulates the relevant context.
WHY NOT A: Near-real-time corpus updates favor Strategy A (recursive splitting) because it is computationally cheap. Strategy B's higher indexing time makes it unsuitable for high-frequency updates.
WHY NOT C: Queries requiring precise single-sentence answers benefit from more granular, fixed-size chunks (Strategy A). Semantic chunking groups multiple sentences together, diluting precision for single-sentence lookups.
WHY NOT D: A short-context-window embedding model struggles with larger semantic chunks. Strategy B would produce chunks that exceed a 256-token model's window, causing truncation and degraded embeddings.