Home / DE Associate practice test / Data Transformation and Modeling / Question

Databricks Data Engineer Associate · free practice question

A silver transformation must drop any row where the `user_id` column is null, while keeping rows that have nulls in other columns. Which PySpark expression…

Exam-style question on Data Transformation and Modeling from the free DE Associate practice test, with the answer and a full explanation below.

Updated November 10, 2026

Data Transformation and Modeling

A silver transformation must drop any row where the user_id column is null, while keeping rows that have nulls in other columns. Which PySpark expression does exactly this?

  1. Adf.filter(col("user_id").isNotNull()), which retains only the rows whose user_id column holds a non-null value while leaving any nulls still present in the other columns of those surviving rows completely untouched.
  2. Bdf.dropna(how="all"), which removes a row only when every one of its columns is null at the same time, so it keeps rows whose user_id is null as long as any other column in that same row still holds a value.
  3. Cdf.na.drop() called with no arguments, which walks every column and removes any row that contains a null in at least one of its fields, so it also discards rows that merely have a null in some column other than user_id.
  4. Ddf.fillna(0, subset=["user_id"]), which finds rows with a null user_id and substitutes the integer zero into that column, so those rows are kept in the output rather than being removed from the DataFrame at all.
  5. Edf.filter(col("user_id") == null), which compares user_id against null and keeps the rows where that equality evaluates to true, thereby selecting exactly the rows whose user_id value is missing from the data.
Answer & explanation

Correct answer: A

WHY A: filter(col("user_id").isNotNull()) keeps only rows with a non-null user_id and leaves nulls in other columns intact — exactly the requirement. WHY NOT C: na.drop() with no subset drops rows with a null in ANY column, removing more than intended. WHY NOT B: dropna(how="all") drops a row only when every column is null, so it fails to remove the null-user_id rows we need gone. WHY NOT D: fillna replaces nulls with 0 rather than dropping the rows. WHY NOT E: == null is never true in Spark; null tests require isNull/isNotNull, so this filter returns no rows.

Take the full DE Associate practice test free →

Would you pass today?

This question is one of 45 in the free, timed Databricks Data Engineer Associate practice test. Take it and get a section-by-section score.

Start the free test →