When inference logging is enabled for a Databricks Model Serving endpoint, a developer observes that the logged request information does not include the document chunks retrieved by a RAG application. Only the final user query and the model's response are captured. What is the fundamental reason for this behavior?
Answer & explanation
Correct answer: A
WHY A: Inference tables log what is actually sent to and returned from the model serving endpoint. If the RAG chain assembles retrieval results internally without including them in the logged request payload, they will not appear in the inference table. The developer must either restructure the chain to include retrieved chunks in the request/response payload (so they are captured in inference logs) or enable MLflow Tracing, which captures intermediate span-level data including retrieval results separately from the inference table.
WHY NOT B: Inference tables are configured at the endpoint level and persist across model version promotions. They are not automatically disabled on promotion and do not require re-enabling.
WHY NOT C: Inference tables capture the serialized request and response payloads at the endpoint's HTTP interface boundary. They do not automatically introspect or capture internal Python state, intermediate function calls, or variables that were not included in those payloads.
WHY NOT D: Both PyFunc and LangChain model flavors support inference logging. The flavor does not determine whether intermediate pipeline data is logged; the payload structure does.