U 2d=@sddlmZddlZddlmZmZddlmZddlm Z ddl m Z m Z m Z dd lmZdd lmZmZmZdd lmZmZmZdd lmZdd lmZddZddZGddde ee ZdS))deepcopyN)IntegralReal) SelectorMixin)_get_feature_importances) BaseEstimatorcloneMetaEstimatorMixin) _safe_tags)check_is_fitted check_scalar _num_features) HasMethodsIntervalOptions)NotFittedError) available_ifc Cs4|dkrz|jj}t|do"|jdk}d|k}d|kobt|drLt|jdpbt|dobt|jd}|sp|sp|rvd }nd }t|t r(d |kr| d \}}t | }| }|d krt |}n |d krt|}n td |||}n8|d krt |}n"|d krt|}n td|nt |}|S)zInterpret the threshold valueNpenaltyl1ZLassoZ ElasticNet l1_ratio_g?l1_ratiogh㈵>mean*medianzUnknown reference: z6Expected threshold='mean' or threshold='median' got %s) __class____name__hasattrrnpiscloserr isinstancestrsplitfloatstriprr ValueError) estimatorZ importances thresholdZest_nameZis_l1_penalizedZis_lassoZis_elasticnet_l1_penalizedZscale referencer*I/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/feature_selection/_from_model.py_calculate_thresholds>           r,cs fddS)zCheck if we can delegate a method to the underlying estimator. First, we check the fitted estimator if available, otherwise we check the unfitted estimator. cs"t|drt|jSt|jS)N estimator_)rr-r'selfattrr*r+Lsz _estimator_has..r*r0r*r0r+_estimator_hasFsr3c @seZdZUdZedgeeddddedgdgeeddddeed dd de ee j e j hgeed dd de dgee gd Z eed <ddd dddddZddZddZd"ddZeddZeedd#ddZeddZd d!ZdS)$SelectFromModelaMeta-transformer for selecting features based on importance weights. .. versionadded:: 0.17 Read more in the :ref:`User Guide `. Parameters ---------- estimator : object The base estimator from which the transformer is built. This can be both a fitted (if ``prefit`` is set to True) or a non-fitted estimator. The estimator should have a ``feature_importances_`` or ``coef_`` attribute after fitting. Otherwise, the ``importance_getter`` parameter should be used. threshold : str or float, default=None The threshold value to use for feature selection. Features whose absolute importance value is greater or equal are kept while the others are discarded. If "median" (resp. "mean"), then the ``threshold`` value is the median (resp. the mean) of the feature importances. A scaling factor (e.g., "1.25*mean") may also be used. If None and if the estimator has a parameter penalty set to l1, either explicitly or implicitly (e.g, Lasso), the threshold used is 1e-5. Otherwise, "mean" is used by default. prefit : bool, default=False Whether a prefit model is expected to be passed into the constructor directly or not. If `True`, `estimator` must be a fitted estimator. If `False`, `estimator` is fitted and updated by calling `fit` and `partial_fit`, respectively. norm_order : non-zero int, inf, -inf, default=1 Order of the norm used to filter the vectors of coefficients below ``threshold`` in the case where the ``coef_`` attribute of the estimator is of dimension 2. max_features : int, callable, default=None The maximum number of features to select. - If an integer, then it specifies the maximum number of features to allow. - If a callable, then it specifies how to calculate the maximum number of features allowed by using the output of `max_features(X)`. - If `None`, then all features are kept. To only select based on ``max_features``, set ``threshold=-np.inf``. .. versionadded:: 0.20 .. versionchanged:: 1.1 `max_features` accepts a callable. importance_getter : str or callable, default='auto' If 'auto', uses the feature importance either through a ``coef_`` attribute or ``feature_importances_`` attribute of estimator. Also accepts a string that specifies an attribute name/path for extracting feature importance (implemented with `attrgetter`). For example, give `regressor_.coef_` in case of :class:`~sklearn.compose.TransformedTargetRegressor` or `named_steps.clf.feature_importances_` in case of :class:`~sklearn.pipeline.Pipeline` with its last step named `clf`. If `callable`, overrides the default feature importance getter. The callable is passed with the fitted estimator and it should return importance for each feature. .. versionadded:: 0.24 Attributes ---------- estimator_ : estimator The base estimator from which the transformer is built. This attribute exist only when `fit` has been called. - If `prefit=True`, it is a deep copy of `estimator`. - If `prefit=False`, it is a clone of `estimator` and fit on the data passed to `fit` or `partial_fit`. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. .. versionadded:: 0.24 max_features_ : int Maximum number of features calculated during :term:`fit`. Only defined if the ``max_features`` is not `None`. - If `max_features` is an `int`, then `max_features_ = max_features`. - If `max_features` is a callable, then `max_features_ = max_features(X)`. .. versionadded:: 1.1 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 threshold_ : float The threshold value used for feature selection. See Also -------- RFE : Recursive feature elimination based on importance weights. RFECV : Recursive feature elimination with built-in cross-validated selection of the best number of features. SequentialFeatureSelector : Sequential cross-validation based feature selection. Does not rely on importance weights. Notes ----- Allows NaN/Inf in the input if the underlying estimator does as well. Examples -------- >>> from sklearn.feature_selection import SelectFromModel >>> from sklearn.linear_model import LogisticRegression >>> X = [[ 0.87, -1.34, 0.31 ], ... [-2.79, -0.02, -0.85 ], ... [-1.34, -0.48, -2.55 ], ... [ 1.92, 1.48, 0.65 ]] >>> y = [0, 1, 0, 1] >>> selector = SelectFromModel(estimator=LogisticRegression()).fit(X, y) >>> selector.estimator_.coef_ array([[-0.3252302 , 0.83462377, 0.49750423]]) >>> selector.threshold_ 0.55245... >>> selector.get_support() array([False, True, False]) >>> selector.transform(X) array([[-1.34], [-0.02], [-0.48], [ 1.48]]) Using a callable to create a selector that can use no more than half of the input features. >>> def half_callable(X): ... return round(len(X[0]) / 2) >>> half_selector = SelectFromModel(estimator=LogisticRegression(), ... max_features=half_callable) >>> _ = half_selector.fit(X, y) >>> half_selector.max_features_ 2 fitNZboth)closedbooleanrightrleftr)r'r(prefit norm_order max_featuresimportance_getter_parameter_constraintsFauto)r(r;r<r=r>cCs(||_||_||_||_||_||_dS)N)r'r(r;r>r<r=)r/r'r(r;r<r=r>r*r*r+__init__s zSelectFromModel.__init__c Cs t|d|j}t|d|j}|jr^zt|jWn,tk r\}ztd|W5d}~XYnXt|rptdn"|dk rt|tst d|dt ||j d|j d}t |||j}|jdk rtj|td }tj| d d d|}d ||<ntj|td }d |||k<|S)Nr- max_features_EWhen `prefit=True`, `estimator` is expected to be a fitted estimator.z[When `prefit=True` and `max_features` is a callable, call `fit` before calling `transform`.z5`max_features` must be an integer. Got `max_features=z ` instead.normr'getterZtransform_funcr<)ZdtypeZ mergesort)kindTF)getattrr'r=r;r rcallabler!rr&rr>r<r,r(rZ zeros_likeboolZargsortZ ones_like)r/r'r=excscoresr(maskZcandidate_indicesr*r*r+_get_support_masksB    z!SelectFromModel._get_support_maskcCsJ|jdk rFt|}t|jr(||}n|j}t|dtd|d||_dS)Nr=r)Zmin_valZmax_val)r=rrIrrrB)r/XZ n_featuresr=r*r*r+_check_max_features1s   z#SelectFromModel._check_max_featuresc Ks||||jrbzt|jWn,tk rR}ztd|W5d}~XYnXt|j|_nt|j|_|jj ||f|t |jdr|jj |_ n|j |dd|S)aFit the SelectFromModel meta-transformer. Parameters ---------- X : array-like of shape (n_samples, n_features) The training input samples. y : array-like of shape (n_samples,), default=None The target values (integers that correspond to classes in classification, real numbers in regression). **fit_params : dict Other estimator specific parameters. Returns ------- self : object Fitted estimator. rCNfeature_names_in_Treset) _validate_paramsrPr;r r'rrr-r r5rrQ_check_feature_names)r/rOy fit_paramsrKr*r*r+r5Cs$    zSelectFromModel.fitcCs&t|j|jd|jd}t|j||jS)z+Threshold value used for feature selection.rDrE)rr-r>r<r,r'r()r/rLr*r*r+ threshold_nszSelectFromModel.threshold_ partial_fitc Kst|d }|r"||||jrx|rtzt|jWn,tk rf}ztd|W5d}~XYnXt|j|_|S|rt |j|_|jj ||f|t|jdr|jj |_ n|j ||d|S)a!Fit the SelectFromModel meta-transformer only once. Parameters ---------- X : array-like of shape (n_samples, n_features) The training input samples. y : array-like of shape (n_samples,), default=None The target values (integers that correspond to classes in classification, real numbers in regression). **fit_params : dict Other estimator specific parameters. Returns ------- self : object Fitted estimator. r-rCNrQrR) rrTrPr;r r'rrr-r rYrQrU)r/rOrVrWZ first_callrKr*r*r+rYys.      zSelectFromModel.partial_fitc CsLz t|Wn6tk rB}ztd|jj|W5d}~XYnX|jjS)z%Number of features seen during `fit`.z*{} object has no n_features_in_ attribute.N)r rAttributeErrorformatrrr-n_features_in_)r/Znfer*r*r+r\s zSelectFromModel.n_features_in_cCsdt|jddiS)N allow_nan)key)r r'r.r*r*r+ _more_tagsszSelectFromModel._more_tags)N)N)r __module__ __qualname____doc__rrrr"rrrinfrIr?dict__annotations__rArNrPr5propertyrXrr3rYr\r_r*r*r*r+r4Ss8  * +  1 r4)copyrZnumpyrZnumbersrr_baserrbaser r r Z utils._tagsr Zutils.validationr rrZutils._param_validationrrr exceptionsrZutils.metaestimatorsrr,r3r4r*r*r*r+s      2