A silver transformation must drop any row where the user_id column is null, while keeping rows that have nulls in other columns. Which PySpark expression does exactly this?
Answer & explanation
Correct answer: A
WHY A: filter(col("user_id").isNotNull()) keeps only rows with a non-null user_id and leaves nulls in other columns intact — exactly the requirement. WHY NOT C: na.drop() with no subset drops rows with a null in ANY column, removing more than intended. WHY NOT B: dropna(how="all") drops a row only when every column is null, so it fails to remove the null-user_id rows we need gone. WHY NOT D: fillna replaces nulls with 0 rather than dropping the rows. WHY NOT E: == null is never true in Spark; null tests require isNull/isNotNull, so this filter returns no rows.