Home / DE Associate practice test / Data Transformation and Modeling / Question

Databricks Data Engineer Associate · free practice question

A bronze table may contain duplicate event rows that are identical across all columns due to at-least-once delivery. The silver job must keep exactly one copy…

Exam-style question on Data Transformation and Modeling from the free DE Associate practice test, with the answer and a full explanation below.

Updated November 10, 2026

Data Transformation and Modeling

A bronze table may contain duplicate event rows that are identical across all columns due to at-least-once delivery. The silver job must keep exactly one copy of each fully-identical row. Which operation is correct and appropriate?

  1. Adf.dropDuplicates(["event_id"]), which is the required form here because Spark can only deduplicate against a single named key column and will raise an error if you try to compare rows across every column at once.
  2. Bdf.groupBy().count(), which groups the entire DataFrame and returns the set of distinct rows alongside a column reporting how many times each fully-identical row occurred in the original bronze input dataset.
  3. Cdf.limit(1), which keeps exactly one row from the DataFrame and discards all the rest, on the reasoning that retaining a single row is sufficient to guarantee that no duplicate event rows remain in the output.
  4. Ddf.dropDuplicates() with no subset, equivalent to df.distinct(), which removes rows that are identical across every one of their columns and keeps a single copy of each fully-identical event row in the result.
  5. Edf.coalesce(1), which merges the DataFrame's partitions down into a single partition so that the duplicate event rows are physically collapsed together and only one copy of each identical row is written out.
Answer & explanation

Correct answer: D

WHY D: With no subset, dropDuplicates() (same as distinct()) removes rows identical across all columns — exactly the requirement for fully-identical duplicates. WHY NOT A: Deduping on event_id alone collapses rows sharing an id but differing elsewhere, and dedup is not limited to one column. WHY NOT B: groupBy().count() returns aggregate counts, not the deduplicated row set in usable form. WHY NOT C: limit(1) keeps one row of the whole DataFrame, discarding real data. WHY NOT E: coalesce(1) only changes partitioning; it does not remove duplicate rows.

Take the full DE Associate practice test free →

Would you pass today?

This question is one of 45 in the free, timed Databricks Data Engineer Associate practice test. Take it and get a section-by-section score.

Start the free test →