Home / GenAI Engineer practice test / Application Development

Free · 8 questions with explanations

Application Development: Databricks Generative AI Engineer Associate Practice Questions

Exam-style questions on Application Development. Pick your answer, then open the explanation to see why it's right — and why the other options are wrong.

Updated November 11, 2026

1 Application Development

A team is building a LangChain pipeline that must: (1) accept a user question, (2) retrieve relevant document chunks, (3) format them into a prompt, and (4) stream tokens to the UI in real time. The runnable map inputs = {'context': retriever, 'question': RunnablePassthrough()} already supplies the prompt's variables. Using LangChain Expression Language (LCEL), which approach correctly enables streaming on the composed chain?

  1. Achain = inputs | prompt | llm.stream() | StrOutputParser(), then call chain.invoke(input) to receive the tokens
  2. Bchain = inputs | prompt | llm | StrOutputParser(), then call chain.stream(input) to iterate over the tokens
  3. Cchain = LLMChain(retriever=retriever, prompt=prompt, llm=llm), then call chain.run(input, stream=True)
  4. Dchain = retriever.invoke() + prompt.format() + llm.predict(), then wrap the call in a streaming loop
Show answer & explanation

Correct answer: B

WHY B is correct: In LCEL, the pipe operator (|) composes Runnable objects into a pipeline. The chain is defined as inputs | prompt | llm | StrOutputParser() and streaming is activated by calling .stream(input) on the composed chain object. The LLM node automatically yields tokens when .stream() is called at the chain level. WHY NOT A: .stream() is a method you call on the finished chain, not a step inside it. llm.stream() needs an input and returns an iterator of chunks, not a Runnable, so it cannot be composed with the pipe operator, and invoke() on a chain returns the full result rather than a token stream. WHY NOT C: LLMChain is the legacy pre-LCEL class. It does not accept retriever= as a constructor parameter and does not support stream=True as a run() keyword argument in the standard API. WHY NOT D: Manually chaining .invoke(), .format(), and .predict() produces sequential blocking synchronous calls. It does not compose a streaming pipeline and provides no real-time token delivery.

2 Application Development

A healthcare AI assistant must refuse any request for personalized medical diagnosis while still helpfully answering general health education questions. Which guardrail implementation is MOST appropriate?

  1. AUse an intent classifier that separates personal diagnosis requests from general health education and routes each to its own response path
  2. BUse a keyword blocklist that rejects prompts containing terms such as diagnose, symptoms or treatment before they reach the model
  3. CSet max_tokens to 50 so that responses stay too short to contain a detailed personal diagnosis or a treatment recommendation
  4. DRemove medical content from the fine-tuning dataset so that the model lacks the knowledge needed to produce a personal diagnosis, then redeploy it
Show answer & explanation

Correct answer: A

WHY A is correct: A classifier-based intent guardrail accurately distinguishes the user's semantic intent — 'I have chest pain, what's wrong with me?' (seeking diagnosis) versus 'How does the heart pump blood?' (seeking general education). This allows the assistant to refuse and redirect diagnosis requests while remaining useful for educational queries. WHY NOT B: A keyword blocklist is overly broad and brittle. Terms like 'symptoms' or 'treatment' appear in legitimate general health education questions. Blocking on keywords would incorrectly refuse a large proportion of benign queries. WHY NOT C: A short max_tokens limit prevents detailed responses but does not prevent the model from partially providing a diagnosis within 50 tokens. It also truncates legitimate general health explanations. WHY NOT D: Removing medical training data degrades the model's ability to answer general health education questions — which is a core use case. This approach is also impractical for foundation models whose pretraining is fixed.

3 Application Development

An ML team evaluates three models for a text-to-SQL task: - Model X: Execution Accuracy=0.88, cost per query=$0.012 - Model Y: Execution Accuracy=0.91, cost per query=$0.047 - Model Z: Execution Accuracy=0.86, cost per query=$0.003 The business constraints require cost per query < $0.015 AND Execution Accuracy > 0.87. Which model meets BOTH criteria?

  1. AModel Y, because it has the highest Execution Accuracy of the three and accuracy matters most for text-to-SQL
  2. BModel Z, because it has the lowest cost per query and its accuracy is within one point of the threshold
  3. CBoth Model X and Model Y qualify, so the team should choose between them based on familiarity
  4. DModel X, because it is the only one above 0.87 accuracy and below $0.015 per query
Show answer & explanation

Correct answer: D

WHY D is correct: Applying both constraints: Execution Accuracy > 0.87 — Model X (0.88 ✓), Model Y (0.91 ✓), Model Z (0.86 ✗). Cost < $0.015 — Model X ($0.012 ✓), Model Y ($0.047 ✗), Model Z ($0.003 ✓). Only Model X satisfies both constraints simultaneously. WHY NOT A: Model Y achieves the highest accuracy but its $0.047 cost is more than 3× the $0.015 budget ceiling. It fails the cost constraint and is ineligible. WHY NOT B: Model Z costs only $0.003 per query but its Execution Accuracy of 0.86 falls below the required 0.87 threshold. It fails the accuracy constraint. WHY NOT C: Model Y fails the cost constraint ($0.047 > $0.015), so it does not qualify. There is only one model that meets both criteria, making team familiarity irrelevant as a selection factor.

4 Application Development

A developer is building a financial research assistant. When a user asks 'Compare the Q3 earnings of NVDA and MSFT', the system extracts company tickers and the time period, fetches the earnings data, then builds a prompt. Which prompt template structure CORRECTLY implements this augmentation pattern?

  1. Af"You are a financial assistant.\n\nFinancial data:\n{all_company_data}\n\nSummarize each of the companies, then answer: {user_input}"
  2. Bf"You are a financial assistant.\n\nUser question: {user_input}\n\nAnswer from what you learned about these companies during your training."
  3. Cf"You are a financial assistant.\n\nTickers: {tickers}\nPeriod: {period}\n\nUser question: {user_input}\n\nDo not use any external data sources."
  4. Df"You are a financial assistant.\n\nRelevant data:\n{retrieved_earnings_data}\n\nUser question: {user_input}\n\nAnswer using only the data provided above."
Show answer & explanation

Correct answer: D

WHY D: This template separates the retrieved, targeted context (earnings data fetched using the extracted tickers NVDA, MSFT and period Q3) from the user question, and explicitly grounds the model with 'Answer using only the data provided above.' This is the canonical retrieval-augmented prompting structure. WHY NOT B: Directing the model to answer from its training data is the opposite of augmentation. Training data may contain stale or incorrect quarterly figures. WHY NOT C: Injecting the ticker symbols and period without the retrieved earnings data leaves the model without factual context. It may still hallucinate specific revenue figures. WHY NOT A: Injecting all company data indiscriminately bloats the context window with irrelevant records and introduces noise that degrades the quality of the targeted comparison.

5 Application Development

A developer logs a LangChain agent to MLflow using the Databricks Agent Framework's code-based logging approach with mlflow.langchain.log_model(lc_model='/path/to/agent.py', ...). Which step must be completed NEXT before the agent can be served at a Databricks Model Serving endpoint?

  1. ACall mlflow.end_run() to finalize the experiment run, which makes the logged model available to Model Serving
  2. BUpload the logged model artifacts to a cloud storage bucket and give Model Serving the bucket URI
  3. CRegister the logged model in Unity Catalog with mlflow.register_model() and a three-level name
  4. DRe-run the agent notebook to regenerate the logged artifacts before they expire and can no longer be registered
Show answer & explanation

Correct answer: C

WHY C: After logging, the agent must be registered to Unity Catalog: set the registry URI to databricks-uc and call mlflow.register_model() with the logged model URI and a three-level catalog.schema.model name. Registration is required before the agent can be deployed to a Model Serving endpoint. WHY NOT A: mlflow.end_run() closes the active experiment run but does not register or deploy anything. A logged model that has not been registered cannot be deployed to Model Serving. WHY NOT B: MLflow stores the logged artifacts in its configured artifact store automatically. A manual upload to cloud storage is not part of the Agent Framework deployment workflow. WHY NOT D: Code-based logging captures the agent's code file to run at serving time; the logged artifacts do not expire. Re-running the notebook is not a deployment step.

6 Application Development

A deployed RAG-based assistant returns the following answer to 'What is the refund policy?': 'Our refund policy allows returns within 60 days of purchase with a full refund, no questions asked.' However, the retrieved source document clearly states that returns are only accepted within 30 days. Which quality issue BEST describes this response?

  1. APrompt injection: the wording of the user's question caused the model to override the document context
  2. BFaithfulness hallucination: the generated claim contradicts the retrieved document
  3. CContext stuffing: too many retrieved documents diluted the passage that held the correct information
  4. DToken limit truncation: the 30-day policy text was cut off before it reached the model's context window
Show answer & explanation

Correct answer: B

WHY B is correct: Faithfulness hallucination occurs when a model generates content inconsistent with or contradictory to the provided retrieval context. The model produced '60 days' despite the retrieved document stating '30 days' — a direct faithfulness violation. WHY NOT A: Prompt injection involves a malicious or cleverly crafted user input that causes the model to bypass its instructions (e.g., 'ignore the above'). The user's question here is direct and benign. WHY NOT C: Context stuffing refers to retrieving too many document chunks that may confuse the model. A single document being contradicted by the model's output is a faithfulness issue, not a retrieval volume issue. WHY NOT D: Token limit truncation results in missing or incomplete information, typically visible as abrupt cut-off generation. Here the model produced a confident, complete — but factually incorrect — answer, which is characteristic of hallucination, not truncation.

7 Application Development

After deploying a Gen AI assistant, the team configures a monitoring pipeline that applies the same LLM judge metrics (faithfulness, toxicity) to a daily sample of real user queries and responses. How does this monitoring activity DIFFER from the pre-deployment offline evaluation phase?

  1. AMonitoring uses a different set of metrics, since faithfulness can be measured in the offline phase but not on production samples
  2. BMonitoring runs continuously on live traffic to detect drift, while offline evaluation is a one-time gate before deployment
  3. CMonitoring does not use LLM judges and measures system-level metrics such as latency and error rates on the sampled traffic instead
  4. DThe two are the same process, and Databricks recommends a single shared pipeline for both phases so that the work is not duplicated
Show answer & explanation

Correct answer: B

WHY B is correct: Offline evaluation is a static, pre-deployment quality gate run on a fixed benchmark dataset to decide whether to release. Online monitoring is continuous and post-deployment, applying evaluation logic (including LLM judges) to live production samples to detect quality regression, data drift, and safety violations over time. The key differences are: data source (static benchmark vs. live traffic), frequency (point-in-time vs. continuous), and purpose (deployment gate vs. production health tracking). WHY NOT A: Databricks explicitly supports using the same LLM judge configuration — including faithfulness — across both offline evaluation and online monitoring. Faithfulness measurement is not restricted to offline use. WHY NOT C: Modern Gen AI monitoring systems (including Databricks Agent Evaluation) use LLM judges to assess quality dimensions well beyond system-level latency and error rates. WHY NOT D: While the same LLM judge configuration is commonly reused across both phases, the processes differ fundamentally in dataset type, cadence, and purpose. Stating they are identical would eliminate the distinction that the Gen AI application lifecycle explicitly makes between them.

8 Application Development

A team is building an internal coding assistant for Python and SQL. The model must generate syntactically correct code, explain code snippets, and suggest refactoring improvements. Which LLM attribute is MOST critical for this application?

  1. AStrong results on code-specific benchmarks such as HumanEval or MBPP for program synthesis
  2. BA recent knowledge cutoff date that brings the latest research paper abstracts into its training data
  3. CHigh scores on conversational empathy and sentiment analysis benchmarks for assistant models
  4. DLow perplexity on long-form fiction datasets, indicating fluent generation of long creative narratives
Show answer & explanation

Correct answer: A

WHY A: For a coding assistant, performance on code benchmarks (HumanEval measures pass@k for synthesized Python functions; MBPP evaluates diverse programming problems) directly predicts the ability to generate syntactically correct, idiomatic code. Code-completion fine-tuning further specializes the model for the exact task. WHY NOT B: Knowledge cutoff primarily matters for factual or current-events tasks. Python and SQL syntax, standard libraries, and best practices evolve slowly; a recent cutoff is far less predictive of code quality than code-specific training. WHY NOT C: Conversational empathy and sentiment analysis are needed for customer-facing emotional support chatbots, which is unrelated to technical code generation. WHY NOT D: Creative narrative generation does not transfer to code synthesis correctness. These are orthogonal model attributes.

Take the full GenAI Engineer practice test →

From the free full-length test

14 more Application Development questions with answers

Each opens on its own page with the correct answer and the full explanation.

All free GenAI Engineer exam questions & answers →