U 3d @sdZddlmZmZddlZddlmZmZddlZ ddl m Z ddl m Z mZmZddl mZdd l mZdd lmZmZdd lmZdd lmZdd lmZddlmZmZddlmZmZddl m!Z!dddgZ"eedkrddl m#Z$n ddl m$Z$ddZ%d.ddZ&dd Z'd/d"d#Z(d$d%Z)Gd&d'd'eeeee ed(Z*Gd)dde*Z+Gd*dde*Z,Gd+d,d,e*Z-Gd-ddeee Z.dS)0zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). )IntegralRealN)ABCMetaabstractmethod)svd) BaseEstimatorRegressorMixinTransformerMixin)MultiOutputMixin)ClassNamePrefixFeaturesOutMixin) check_arraycheck_consistent_length) sp_version) parse_version)svd_flip)check_is_fitted FLOAT_DTYPES)Interval StrOptions)ConvergenceWarning PLSCanonical PLSRegressionPLSSVDz1.7)pinv)pinv2c Cst|ddd\}}}|jj}ddd}t|||t|j}t||k}|ddd|f}||d|}t t t ||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssumZ transpose conjugatedot)ausZvhtZfactorZcondZrankr.D/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/cross_decomposition/_pls.py _pinv2_old&s  r0Aư>Fc st|jjztfdd|jD}Wn,tk rV}ztd|W5d}~XYnXd}|dkrvt|t|} } t|D]} |dkrt | |} nt |j|t ||} | t t | | } t || } |dkrt | | }nt |j| t | j| }|r*|t t ||}t ||t ||}| |}t |||ksp|j ddkrvq|| }q~| d}||krt dt| ||fS) a?Return the first left and right singular vectors of X'Y. Provides an alternative to the svd(X'Y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3s&|]}tt|kr|VqdSN)r#anyabs).0colr&r.r/ Esz;_get_first_singular_vectors_power_method..Y residual is constantNdBz$Maximum number of iterations reached)r#r%r r&nextT StopIterationr0ranger)sqrtshapewarningswarnr)XYmodemax_itertolnorm_y_weightsZy_scoreeZ x_weights_oldZX_pinvZY_pinvi x_weightsZx_score y_weightsZx_weights_diffZn_iterr.r9r/(_get_first_singular_vectors_power_method8s8   "  rQcCs@t|j|}t|dd\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'Y. Here the whole SVD is computed. FrNr)r#r)r@r)rGrHCU_Vtr.r.r/_get_first_singular_vectors_svdssrWTcCs|jdd}||8}|jdd}||8}|rr|jddd}d||dk<||}|jddd}d||dk<||}n t|jd}t|jd}||||||fS)z{Center X, Y and scale if the scale parameter==True Returns ------- X, Y, x_mean, y_mean, x_std, y_std raxisr>)rYZddofg?)ZmeanZstdr#ZonesrD)rGrHscaleZx_meanZy_meanZx_stdZy_stdr.r.r/_center_scale_xy}s     r\cCs2tt|}t||}||9}||9}dS)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r#Zargmaxr6sign)r+vZbiggest_abs_val_idxr]r.r.r/ _svd_flip_1dsr_c @seZdZUdZeeddddgdgeddhged d hged d hgeeddddgeed dddgdgdZe e d<e d%ddd d ddddddZ ddZ d&ddZd'ddZd(ddZd)dd Zed!d"Zd#d$ZdS)*_PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf r>Nleftclosedboolean regression canonicalr1r=rnipalsr n_componentsr[deflation_moderI algorithmrJrKcopy_parameter_constraintsrTr2r3)r[rjrIrkrJrKrlc Cs4||_||_||_||_||_||_||_||_dSr4)rirjrIr[rkrJrKrl) selfrir[rjrIrkrJrKrlr.r.r/__init__s z _PLS.__init__c Cs|t|||j|tj|jdd}t|dtj|jdd}|jdkrT|dd}|j d}|j d}|j d}|j }|j d kr|n t |||}||krt d |d |d |j d k|_|j}t|||j\} } |_|_|_|_t||f|_t||f|_t||f|_t||f|_t||f|_t||f|_g|_t| jj} t |D]} |j!dkr"tj"t#| d| kdd} d| dd| f<z$t$| | |j%|j&|j'|d\}}}WnPt(k r}z0t)|dkrt*+d| WY qzW5d}~XYnX|j,|n|j!dkrrre`n_components` upper bound is . Got instead. Reduce `n_components`.rfrg rXrZN)rIrJrKrLr;z$Y residual is constant at iteration r)r)8_validate_paramsr_validate_datar#float64rlr ndimreshaperDrirjmin ValueErrorZ_norm_y_weightsr\r[_x_mean_y_mean_x_std_y_stdzeros x_weights_ y_weights_ _x_scores _y_scores x_loadings_ y_loadings_n_iter_r%r r&rBrkallr6rQrIrJrKrAstrrErFappendrWr_r)outerrr@ x_rotations_ y_rotations__coef_ intercept__n_features_out)rnrGrHnpqrirank_upper_boundrLZXkZYkZY_epskZYk_maskrOrPrrMx_scoresZy_ssy_scoresZ x_loadingsZ y_loadingsr.r.r/fits                z_PLS.fitcCst||j||tdd}||j8}||j}t||j}|dk rt|dd|td}|j dkrl| dd}||j 8}||j }t||j }||fS|S)a.Apply the dimension reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to transform. Y : array-like of shape (n_samples, n_targets), default=None Target vectors. copy : bool, default=True Whether to copy `X` and `Y`, or perform in-place normalization. Returns ------- x_scores, y_scores : array-like or tuple of array-like Return `x_scores` if `Y` is not given, `(x_scores, y_scores)` otherwise. Frlr resetNrH)rsrtrlr r>ru)rr{rrrr#r)rr r}r~rrr)rnrGrHrlrrr.r.r/ transformis(      z_PLS.transformcCst|t|dtd}t||jj}||j9}||j7}|dk r|t|dtd}t||j j}||j 9}||j 7}||fS|S)aeTransform data back to its original space. Parameters ---------- X : array-like of shape (n_samples, n_components) New data, where `n_samples` is the number of samples and `n_components` is the number of pls components. Y : array-like of shape (n_samples, n_components) New target, where `n_samples` is the number of samples and `n_components` is the number of pls components. Returns ------- X_reconstructed : ndarray of shape (n_samples, n_features) Return the reconstructed `X` data. Y_reconstructed : ndarray of shape (n_samples, n_targets) Return the reconstructed `X` target. Only returned when `Y` is given. Notes ----- This transformation will only be exact if `n_components=n_features`. rG)rsr NrH) rr rr#matmulrr@rrrrr)rnrGrHZX_reconstructedZY_reconstructedr.r.r/inverse_transforms    z_PLS.inverse_transformcCsDt||j||tdd}||j8}||j}||jj}||jS)aUPredict targets of given samples. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples. copy : bool, default=True Whether to copy `X` and `Y`, or perform in-place normalization. Returns ------- y_pred : ndarray of shape (n_samples,) or (n_samples, n_targets) Returns predicted values. Notes ----- This call requires the estimation of a matrix of shape `(n_features, n_targets)`, which may be an issue in high dimensional space. Fr)rr{rrrrr@r)rnrGrlZYpredr.r.r/predicts    z _PLS.predictcCs|||||S)aLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples, n_targets), default=None Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : ndarray of shape (n_samples, n_components) Return `x_scores` if `Y` is not given, `(x_scores, y_scores)` otherwise. rrrnrGyr.r.r/ fit_transformsz_PLS.fit_transformcCs0t|dr(t|ddr(tdtd|_|jjS)z%The coefficients of the linear model.r _coef_warningTzThe attribute `coef_` will be transposed in version 1.3 to be consistent with other linear models in scikit-learn. Currently, `coef_` has a shape of (n_features, n_targets) and in the future it will have a shape of (n_targets, n_features).F)hasattrgetattrrErF FutureWarningrrr@rnr.r.r/coef_sz _PLS.coef_cCs dddS)NTF)Z poor_scoreZ requires_yr.rr.r.r/ _more_tagssz_PLS._more_tags)r)NT)N)T)N)__name__ __module__ __qualname____doc__rrrrrmdict__annotations__rrorrrrrpropertyrrr.r.r.r/r`s<       ' ,   r`) metaclasscs`eZdZUdZejZeed<dD]Ze eq"d dddddfd d Z fd d Z Z S)ra PLS regression. PLSRegression is also known as PLS2 or PLS1, depending on the number of targets. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, Y) PLSRegression() >>> Y_pred = pls2.predict(X) rmrjrIrkrTr2r3r[rJrKrlc s tj||ddd|||ddS)Nrer1rgrhsuperrornrir[rJrKrl __class__r.r/rotszPLSRegression.__init__cs"t|||j|_|j|_|S)rp)rrrZ x_scores_rZ y_scores_)rnrGrHrr.r/rszPLSRegression.fit)r) rrrrr`rmrrparampopror __classcell__r.r.rr/rs b csVeZdZUdZejZeed<dD]Ze eq"d dddddd fd d Z Z S) ra Partial Least Squares transformer and regressor. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, Y) PLSCanonical() >>> X_c, Y_c = plsca.transform(X, Y) rm)rjrIrTrgr2r3)r[rkrJrKrlc s tj||dd||||ddS)Nrfr1rhr)rnrir[rkrJrKrlrr.r/ros zPLSCanonical.__init__)r rrrrr`rmrrrrrorr.r.rr/rs _ csTeZdZUdZejZeed<dD]Ze eq"d dddddfd d Z Z S) CCAap Canonical Correlation Analysis, also known as "Mode B" PLS. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `Y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `Y`. coef_ : ndarray of shape (n_features, n_targets) The coefficients of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `Y` is approximated as `Y = X @ coef_ + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> Y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, Y) CCA(n_components=1) >>> X_c, Y_c = cca.transform(X, Y) rmrrTr2r3rc s tj||ddd|||ddS)Nrfr=rgrhrrrr.r/roysz CCA.__init__)rrr.r.rr/rs W rc@sfeZdZUdZeeddddgdgdgdZeed<dd d d d d Z ddZ dddZ dddZ dS)raPartial Least Square SVD. This transformer simply performs a SVD on the cross-covariance matrix `X'Y`. It is able to project both the training data `X` and the targets `Y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `Y`. copy : bool, default=True Whether to copy `X` and `Y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> Y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, Y) >>> X_c, Y_c = pls.transform(X, Y) >>> X_c.shape, Y_c.shape ((4, 2), (4, 2)) r>Nrarbrdrir[rlrmrT)r[rlcCs||_||_||_dSr4r)rnrir[rlr.r.r/roszPLSSVD.__init__c Cs*|t|||j|tj|jdd}t|dtj|jdd}|jdkrT|dd}|j }t |j d|j d|j d}||krt d |d |d t |||j\}}|_|_|_|_t|j|}t|dd \}}}|d d d |f}|d |}t||\}}|j} ||_| |_|jj d|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. Y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. rrqrHFrrr>rurrvrwrxrRN)rzrr{r#r|rlr r}r~rirrDrr\r[rrrrr)r@rrrrr) rnrGrHrirrSrTr,rVVr.r.r/rsL    z PLSSVD.fitcCst||j|tjdd}||j|j}t||j}|dk rt|ddtjd}|j dkrh| dd}||j |j }t||j }||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. Y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. F)r rNrH)rsrtr r>ru)rr{r#r|rrr)rr r}r~rrr)rnrGrHZXrrZYrrr.r.r/rs  zPLSSVD.transformcCs|||||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `Y is not None`, `(X_transformed, Y_transformed)` otherwise. rrr.r.r/r/szPLSSVD.fit_transform)r)N)N) rrrrrrrmrrrorrrr.r.r.r/rs D8 )r1r2r3F)T)/rZnumbersrrrEabcrrZnumpyr#Z scipy.linalgrbaserr r r r utilsr rZ utils.fixesrrZ utils.extmathrZutils.validationrrZutils._param_validationrr exceptionsr__all__rrr0rQrWr\r_r`rrrrr.r.r.r/sX           ;  hk