U 2d/@sdZddlmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZmZmZmZdd lmZdd lmZdd lmZdd lmZGd ddee e ZdS)z Sequential feature selection )IntegralRealN) SelectorMixin) BaseEstimatorMetaEstimatorMixinclone) HasMethodsHiddenInterval StrOptions) _safe_tags)check_is_fitted)cross_val_score)get_scorer_namesc @seZdZUdZedggeddhdhdeedddd eedd d d e d gd eedd d d ged d hgd ee e e gdgd egdZ eed<dd d d dd dddZdddZddZddZddZd S)SequentialFeatureSelectoraTransformer that performs Sequential Feature Selection. This Sequential Feature Selector adds (forward selection) or removes (backward selection) features to form a feature subset in a greedy fashion. At each stage, this estimator chooses the best feature to add or remove based on the cross-validation score of an estimator. In the case of unsupervised learning, this Sequential Feature Selector looks only at the features (X), not the desired outputs (y). Read more in the :ref:`User Guide `. .. versionadded:: 0.24 Parameters ---------- estimator : estimator instance An unfitted estimator. n_features_to_select : "auto", int or float, default='warn' If `"auto"`, the behaviour depends on the `tol` parameter: - if `tol` is not `None`, then features are selected until the score improvement does not exceed `tol`. - otherwise, half of the features are selected. If integer, the parameter is the absolute number of features to select. If float between 0 and 1, it is the fraction of features to select. .. versionadded:: 1.1 The option `"auto"` was added in version 1.1. .. deprecated:: 1.1 The default changed from `None` to `"warn"` in 1.1 and will become `"auto"` in 1.3. `None` and `'warn'` will be removed in 1.3. To keep the same behaviour as `None`, set `n_features_to_select="auto" and `tol=None`. tol : float, default=None If the score is not incremented by at least `tol` between two consecutive feature additions or removals, stop adding or removing. `tol` is enabled only when `n_features_to_select` is `"auto"`. .. versionadded:: 1.1 direction : {'forward', 'backward'}, default='forward' Whether to perform forward selection or backward selection. scoring : str or callable, default=None A single str (see :ref:`scoring_parameter`) or a callable (see :ref:`scoring`) to evaluate the predictions on the test set. NOTE that when using a custom scorer, it should return a single value. If None, the estimator's score method is used. cv : int, cross-validation generator or an iterable, default=None Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross validation, - integer, to specify the number of folds in a `(Stratified)KFold`, - :term:`CV splitter`, - An iterable yielding (train, test) splits as arrays of indices. For integer/None inputs, if the estimator is a classifier and ``y`` is either binary or multiclass, :class:`StratifiedKFold` is used. In all other cases, :class:`KFold` is used. These splitters are instantiated with `shuffle=False` so the splits will be the same across calls. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. n_jobs : int, default=None Number of jobs to run in parallel. When evaluating a new feature to add or remove, the cross-validation procedure is parallel over the folds. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_features_to_select_ : int The number of features that were selected. support_ : ndarray of shape (n_features,), dtype=bool The mask of selected features. See Also -------- GenericUnivariateSelect : Univariate feature selector with configurable strategy. RFE : Recursive feature elimination based on importance weights. RFECV : Recursive feature elimination based on importance weights, with automatic selection of the number of features. SelectFromModel : Feature selection based on thresholds of importance weights. Examples -------- >>> from sklearn.feature_selection import SequentialFeatureSelector >>> from sklearn.neighbors import KNeighborsClassifier >>> from sklearn.datasets import load_iris >>> X, y = load_iris(return_X_y=True) >>> knn = KNeighborsClassifier(n_neighbors=3) >>> sfs = SequentialFeatureSelector(knn, n_features_to_select=3) >>> sfs.fit(X, y) SequentialFeatureSelector(estimator=KNeighborsClassifier(n_neighbors=3), n_features_to_select=3) >>> sfs.get_support() array([ True, False, True, True]) >>> sfs.transform(X).shape (150, 3) fitautowarn) deprecatedrrright)closedNZneitherforwardbackwardZ cv_object estimatorn_features_to_selecttol directionscoringcvn_jobs_parameter_constraints)rrrr r!r"cCs.||_||_||_||_||_||_||_dSNr)selfrrrrr r!r"r'I/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/feature_selection/_sequential.py__init__s z"SequentialFeatureSelector.__init__cCs||jdkrtdt|}|j|dd|dd d}|jd}d |j}|jdkr|j d k rtt d |d|_ n~|jd kr|j d k r|d|_ q|d|_ nRt |jt rd |jkr|ksnt ||j|_ nt |jtrt||j|_ t|j}tj|td}|jd ks.|jdkr4|j n||j }tj } |j d k oZ|jd k} t|D]B} |||||\} } | r| | |j krq| } d|| <qd|jdkr|}||_|j|_ |S)a Learn the features to select from X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples,), default=None Target values. This parameter may be ignored for unsupervised learning. Returns ------- self : object Returns the instance itself. )rNaLeaving `n_features_to_select` to None is deprecated in 1.0 and will become 'auto' in 1.3. To keep the same behaviour as with None (i.e. select half of the features) and avoid this warning, you should manually set `n_features_to_select='auto'` and set tol=None when creating an instance.Zcscr allow_nanT)Z accept_sparseZensure_min_featuresZforce_all_finiterzn_features_to_select must be either 'auto', 'warn', None, an integer in [1, n_features - 1] representing the absolute number of features, or a float in (0, 1] representing a percentage of features to select. Got Nz4tol is only enabled if `n_features_to_select='auto'`rr)shapeZdtyperr)Z_validate_paramsrwarningsr FutureWarningZ _get_tagsZ_validate_datagetr+r ValueErrorZn_features_to_select_ isinstancerrintr rnpzerosboolrinfrange_get_best_new_feature_scoresupport_sum)r&XytagsZ n_features error_msgZcloned_estimator current_maskZ n_iterationsZ old_scoreZis_auto_select_new_feature_idxZ new_scorer'r'r(rsp                     zSequentialFeatureSelector.fitc st|}i|D]V}|}d||<|jdkr8|}|dd|f}t||||j|j|jd|<qt fddd} | | fS)NTr)r!r r"cs|Sr%r') feature_idxZscoresr'r(3zGSequentialFeatureSelector._get_best_new_feature_score..key) r2Z flatnonzerocopyrrr!r r"Zmeanmax) r&rr:r;r>Zcandidate_feature_indicesrAZcandidate_maskZX_newr@r'rBr(r7s$  z5SequentialFeatureSelector._get_best_new_feature_scorecCst||jSr%)rr8r&r'r'r(_get_support_mask6sz+SequentialFeatureSelector._get_support_maskcCsdt|jddiS)Nr*rE)rrrIr'r'r( _more_tags:s z$SequentialFeatureSelector._more_tags)N)__name__ __module__ __qualname____doc__r r r rrr setrcallabler#dict__annotations__r)rr7rJrKr'r'r'r(rs2     fr)rOZnumbersrrZnumpyr2r,_baserbaserrr Zutils._param_validationr r r r Z utils._tagsrZutils.validationrZmodel_selectionrZmetricsrrr'r'r'r(s