Home / DE Professional practice test / Data Transformation, Cleansing, and Quality / Question

Databricks Data Engineer Professional · free practice question

A Lakeflow Spark Declarative Pipelines (DLT) pipeline ingests customer records. The team wants to enforce that `customer_id IS NOT NULL` and `age BETWEEN 0 AND…

Exam-style question on Data Transformation, Cleansing, and Quality from the free DE Professional practice test, with the answer and a full explanation below.

Updated November 16, 2026

Data Transformation, Cleansing, and Quality Easy

A Lakeflow Spark Declarative Pipelines (DLT) pipeline ingests customer records. The team wants to enforce that customer_id IS NOT NULL and age BETWEEN 0 AND 120. Invalid records should be excluded from the curated table (not written), but the pipeline should continue processing valid records and produce data quality metrics for how many were dropped. Which expectation action/operator is the best fit?

  1. AUse @dlt.expect('valid_customer', 'customer_id IS NOT NULL AND age BETWEEN 0 AND 120') because expect drops invalid rows by default and logs metrics.
  2. BUse a Python UDF inside the expectation clause to validate multiple fields; DLT expectations allow custom Python functions inside constraints.
  3. CUse @dlt.expect_or_fail('valid_customer', 'customer_id IS NOT NULL AND age BETWEEN 0 AND 120') so the update is rolled back and metrics are recorded for debugging.
  4. DUse @dlt.expect_or_drop('valid_customer', 'customer_id IS NOT NULL AND age BETWEEN 0 AND 120') so invalid rows are dropped before write and drop counts are logged.
Answer & explanation

Correct answer: D

WHY D: expect_or_drop corresponds to the “drop on violation” behavior: invalid rows are dropped before writing, and the pipeline records metrics (including dropped counts) for observability. WHY NOT A: expect (warn/default) retains invalid records in the target while logging metrics; it doesn’t exclude them. WHY NOT C: expect_or_fail fails the update when invalid records are detected; in addition, when a flow fails, metrics are not recorded for that failed update in the same way as warn/drop. WHY NOT B: Expectation constraints must be valid SQL boolean expressions and do not allow custom Python functions or external calls in the constraint logic.

Take the full DE Professional practice test free →

Would you pass today?

This question is one of 60 in the free, timed Databricks Data Engineer Professional practice test. Take it and get a section-by-section score.

Start the free test →