Description
This issue is to track the work we need to do before we can merge sample-props
branch into main
:
- Based on this prototype: [WIP] sample props (proposal 4) #16079
- Merge Base sample-prop implementation and docs (alternative to #21284) #22083 into
sample-props
. This PR only touchesBaseEstimator
and hence consumers. It does NOT touch meta-estimators, scorers or cv splitters. - Work on splitters and merge that into
sample-props
: FEAT add metadata routing to splitters #22765 - feature flag: SLEP006: introduction of metadata routing through a feature flag #26045: FEAT add SLEP006 with a feature flag #26103
- Add new terms to the glossary: router, consumer, metadata: DOC add metadata routing terms to glossary #26685
- Work on scorers and merge that into
sample-props
(note that this involves an ongoing discussion on whether we'd like to mutate a scorer or not): FEAT SLEP6: scorers #22757 - Get backward compatibility mechanisms in place for meta-estimators which already route given metadata: FEAT multioutput routes metadata #22986
- Work on meta-estimators: this involves writing
get_metadata_routing
in easy cases, and a whole lot more in cases where we'd like to keep backward compatibility in parsing input args suchs asestimator__param
inPipeline
- Once all the above is merged into
sample-props
, do a few tests with third party estimators to see if they'd work out of the box. Note that consumer estimators should work out of the box as long as they inherit fromBaseEstimator
and theirfit
accepts metadata as explicit arguments rather than**kwargs
.- Merged w/o completion of meta-estimators, they got done progressively.
- Check whether a library such as cuML could vendor
_metadata_requests.py
and work with scikit-learn meta-estimators w/o depending on the library.- dropped the idea of supporting vendoring, they'd need to inherit from
BaseEstimator
- dropped the idea of supporting vendoring, they'd need to inherit from
- Refactor tests: Refactor metadata routing classes used in tests #23918
- Merge
sample-props
intomain
: FEAT SLEP006: metadata routing infrastructure #24027
Enhancements:
- ENH make sure warn_on errors on invalid child #24023
- FEA SLEP006: debugging tool for sample props #18936
- RFC Revisiting meta-routing developer API for defining method mappings #28467
Cleanup
Open issues:
- SLEP006 - metadata handling: fit, transform, fit_transform #22987
- SLEP006 - equal requests? {fit, partial_fit}, {predict, predict_proba, predict_log_proba, decision_function} #22988: ENH allow all main top level methods to have a corresponding set metadata request #23342
- RFC SLEP006: allow users to enable a "strict" mode in metadata routing #23920
- RFC SLEP006: verbose vs non-verbose declaration in meta-estimator #23928
- global flags: SLEP006: globally setting request values #26050
- auto-routing: SLEP006: default routing #26179
- SLEP006: mypy can't find
set_{method}_request
methods #23933 - RFC: Breaking Changes for Version 2 #25776: MNT SLEP006: Remove backward compatibility from meta-estimators #25852 - resolution is to use the feature flag instead of breaking change.
- Routing metadata to the
response_method
used by a scorer #27977
Our plan is to hopefully have this feature in 1.1, which we should be releasing in late April/early May.
cc @jnothman @thomasjpfan @lorentzenchr
Misc:
- MNT SLEP006: change release version to 2.0 #25851
- MNT SLEP6: raise NotImplementedError for meta-estimators not supporting metadata routing #27389
From #24027
Once we're finished with merging PRs to sample-props
branch, we should merge this.
I've opened this for us to have an idea of what the whole project is touching.
We probably should rebase and merge this instead of squash and merge.
This closes and fixes the following issues and PRs:
-
Routing issues:
Enables Add TimeSeriesCV and HomogeneousTimeSeriesCV #6322, -
AdaBoost:
Fixes Make Pipeline compatible with AdaBoost #2630,
Fixes fit_params for Adaboost estimator #21706 -
ClassifierChain:
Fixes ClassifierChain does not support GroupKFold #11429, -
FeatureUnion:
Fixesfit_params
in conjunction withFeatureUnion
#7136 -
LogisticRegressionCV:
Fixes LogisticRegressionCV not compatible with LeaveOneGroupOut #8950, -
Multioutput*
Fixes MultiOutputRegressor: Support for more fit parameters #15953 -
OneVsRestClassifier
Fixes OneVsRestClassifier's fit method does not accept kwargs #10882 -
RFE/RFECV:
Fixes RFE/RFECV doesn't work with sample weights #7308 -
*SearchCV:
Fixes Nested CV of LeaveOneGroupOut fails in permutation_test_score #8127,
Fixes grid_search: feeding parameters to scorer functions #8158, -
cross_val*:
Fixes Weighted scoring in cross validation (Closes #4632) #13432,
Fixes Should cross-validation scoring take sample-weights into account? #4632,
Fixes GroupKFold fails in nested cross-validation (similar to #2879) #7646,
Fixesmodel_selection.cross_validate
doesn't passgroups
argument to estimator #20349 -
Stacking*
Fixes Support fit_params in stacking #18028, -
Voting*
Fixes Enable mixed ensembles with estimators that do & don't accept the sample_weight fit_param #20167
Metadata
Metadata
Assignees
Type
Projects
Status
No status
Activity