A team is building a gradient boosting classifier (GBTClassifier in Spark MLlib) to predict customer churn. The dataset includes a zip_code column with 500 distinct values. A colleague suggests one-hot encoding zip_code before training. Which statement best explains why one-hot encoding is potentially problematic for this specific combination of feature and model type?
Answer & explanation
Correct answer: C
WHY C: High-cardinality one-hot encoding (500 categories -> 499 binary columns) causes three compounding problems for tree-based models: (1) massive feature space expansion slows training and increases memory usage, (2) each binary split can only separate 'zip code X vs. all others' rather than grouping geographically similar areas, reducing the information gain of each split, and (3) most columns are near-zero-variance, so trees waste split evaluations on uninformative features. Integer label encoding or target encoding typically works better for tree models with high-cardinality categorical features.
WHY NOT A: Tree-based models do not require one-hot encoding; GBTClassifier can operate on integer-encoded categorical features and does not raise type errors on them.
WHY NOT B: Decision trees do not perform matrix factorization or inversion; rank deficiency is a concern for linear regression, not for tree ensembles.
WHY NOT D: Tree-based models are scale-invariant and do not require feature standardization; StandardScaler is irrelevant for gradient boosting.
WHY NOT E: Null handling and OHE are independent issues; the all-zero vector does represent the dropped category in OHE, but that does not cause trees to over-allocate leaf nodes to null rows.