Home / DE Professional practice test / Developing Code for Data Processing using Python and SQL / Question

Databricks Data Engineer Professional · free practice question

A senior data engineer is designing a new production Python project to be deployed via Databricks Asset Bundles across three environments: `dev`, `staging`…

Exam-style question on Developing Code for Data Processing using Python and SQL from the free DE Professional practice test, with the answer and a full explanation below.

Updated November 16, 2026

Developing Code for Data Processing using Python and SQL Medium

A senior data engineer is designing a new production Python project to be deployed via Databricks Asset Bundles across three environments: dev, staging, and prod. The team consists of 8 engineers who each need to run isolated unit tests locally before pushing code. The project includes shared utility functions used by multiple pipeline notebooks, a set of Lakeflow Spark Declarative Pipeline definitions, and environment-specific configuration values. The engineer wants to ensure the structure is modular, testable, and DAB-compatible. Which project structure and DAB design pattern best satisfies all of these requirements?

  1. APlace the Python logic inside a single main.py file at the project root, with environment-specific configuration values hardcoded as Python constants at the top of the file. Reference this file from each DAB job task. Use a tests/ directory with pytest for unit testing. Store the Lakeflow Spark Declarative Pipelines SQL directly inside the databricks.yml resource block as inline strings to keep configuration in one file.
  2. BPackage the pipeline code into a single Python wheel file that encapsulates each environment's configuration at build time. Use a CI/CD step that rebuilds the wheel for each target environment before deployment. This approach provides environment isolation at the artifact level, avoids the risk of variable substitution errors at deploy time, and keeps the databricks.yml file minimal and free of environment-specific logic.
  3. CStore the logic inside Databricks notebooks in the workspace and use DAB's include: directive to reference workspace notebook paths directly. Deploy configuration differences between environments by maintaining three separate databricks.yml files — one per environment — and manually switching between them during deployment. This avoids the complexity of variable substitution while still using DAB for deployment automation.
  4. DOrganize shared utility functions into a Python package (e.g., src/myproject/utils/) installable as a local wheel, store pipeline definitions as separate .py or .sql files under a pipelines/ directory, define environment-specific values using DAB ${var.*} substitution in databricks.yml with per-target overrides, and maintain a tests/ directory with pytest that imports from the src/ package for local unit testing without requiring a live Databricks cluster.
  5. EUse DAB's built-in Jinja2 templating support to generate environment-specific notebook files by rendering templates at deploy time with databricks bundle render. Store base templates in a templates/ directory and rendered outputs in a generated/ directory. This pattern is the recommended DAB approach for multi-environment projects, as it gives engineers full control over the final deployed artifact content.
Answer & explanation

Correct answer: D

WHY D: This is the canonical scalable DAB project structure. A src/ installable package keeps shared logic modular and locally testable via pip install -e without a live cluster. Separate pipeline definition files keep notebook/pipeline logic decoupled from deployment config. DAB variable substitution (${var.*}) with per-target overrides in databricks.yml is the idiomatic way to handle environment differences without duplicating resource definitions. A tests/ directory with pytest enables fast, cluster-free unit testing.

WHY NOT A: Hardcoding environment-specific values as Python constants makes multi-environment deployment impossible without code changes and violates separation of concerns, and a single main.py is not modular. Inline pipeline SQL inside databricks.yml is not supported.

WHY NOT C: Maintaining three separate databricks.yml files is an anti-pattern that duplicates the resource definitions per environment and introduces drift risk. DAB variable substitution and targets exist precisely to avoid this, and notebook-only logic is hard to unit test locally.

WHY NOT B: Rebuilding a wheel per environment embeds configuration inside an artifact, which is an anti-pattern. Configuration should be external and injected at deploy/runtime, not baked into binary artifacts.

WHY NOT E: DAB does not support Jinja2 templating or a databricks bundle render command. Variable substitution is handled natively by the DAB YAML engine, not via a template rendering step.

Take the full DE Professional practice test free →

Would you pass today?

This question is one of 60 in the free, timed Databricks Data Engineer Professional practice test. Take it and get a section-by-section score.

Start the free test →