A data scientist is preparing features for a linear regression model. The dataset contains a subscription_tier column with 3 nominal categories: 'Bronze', 'Silver', 'Gold'. There is no natural ordering among these tiers for this model's purpose. Which encoding strategy is most appropriate, and why?
Answer & explanation
Correct answer: E
WHY E: One-hot encoding is the appropriate strategy for nominal categorical features in linear regression. Integer encoding (0, 1, 2) would make the model learn a single slope implying that Bronze to Silver to Gold is a uniform linear increase, a false ordinal assumption. OHE creates independent binary indicators, so the model learns a separate additive effect for each tier. With 3 categories and dropLast=True, 2 binary columns are produced, which is sufficient to represent all 3 tiers.
WHY NOT A: Assigning 1/2/3 to nominal categories introduces a false ordinal relationship; linear regression treats the numeric gap between categories as a meaningful slope, which is incorrect for unordered data.
WHY NOT B: Spark MLlib's LinearRegression requires all features to be numeric; passing a string column raises a runtime error, and there is no internal string-to-embedding conversion.
WHY NOT C: Target encoding can be effective, but the blanket claim that it outperforms OHE for linear models is false; it can introduce data leakage if not implemented with cross-fitting, and OHE is the standard reliable approach.
WHY NOT D: StringIndexer alone introduces the same false ordinal assumption; even with 3 categories the model treats the codes 0, 1, and 2 as numerically meaningful and learns incorrect coefficient magnitudes.