Sleeping on the issue, here is my two cents. Feel free to ignore some / of all my remarks if irrelevant:

categorial_features could be None, an array of integers with the indices of the categorial features or an array of booleans indicating which features are categorical. Would it be out of scope to have an "auto" value to infer which values are categorical? Since HGBT is usually to apply to large data sets, it could be too costly. sklearn.preprocessing.OrdinalEncoder has an "auto" option (which is the default value).
If I understood correctly what LightGBM does, the idea is to split the categories into two groups. To do so, the histogram of the categorical feature is computed then sorted, and the Fisher theorem states that the two groups are contiguous. So after this transformation, the feature can be treated as any binned continuous feature.
Are categorical features subject to the max_bins parameter? It could be tricky.
~~For the first node of each tree, the histogram and its sorting can be computed only once, since it will always be the same, right?~~ Edit: No.
How to deal with categories not seen when fitting the tree? Do we need to add a new bin for this case like it is done for missing values? This would lead to max_bins being lower than or equal to 254. This scenario may occur many times:
- For child nodes, a subset of the samples is used, so some categories will no longer be present in this subset but they will be present for new samples when making predictions.
- Rare categories when doing cross-validation.
- Categories present in the test set that were not present in the training set.
Do we allow users to provide categorical features in varying formats (str) and use sklearn.preprocessing.OrdinalEncoder to encode them internally, or do we force the users to make the preprocessing themselves? One of first thing in fit is to call check_X_y.

Uh oh!

Support categorical features in HistGradientBoosting estimators #15550

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions