U 3dq @s"dZddlmZmZddlZddlmZmZm Z ddl m Z m Z ddl mZddlmZmZmZmZdd lmZmZdd lmZdd lmZmZdd lmZdd lmZddl m!Z!ddl m"Z"ddl#m$Z$d%ddZ%d&ddZ&d'ddZ'dddddddddd d!d"Z(Gd#d$d$eeeeZ)dS)(zLocally Linear Embedding)IntegralRealN)svdqrsolve)eye csr_matrix)eigsh) BaseEstimatorTransformerMixin_UnstableArchMixinClassNamePrefixFeaturesOutMixin)check_random_state check_array)_init_arpack_v0)Interval StrOptions)_eigh) stable_cumsum)check_is_fitted) FLOAT_DTYPES)NearestNeighborsMbP?cCst|td}t|td}t|td}|j\}}|jd|ks@ttj||f|jd}tj||jd}t |D]\}} || } | ||} t | | j } t | } | dkr|| }n|}| j dd|d|7<t| |dd}|t|||ddf<ql|S)aCompute barycenter weights of X from Y along the first axis We estimate the weights to assign to each point in Y[indices] to recover the point X[i]. The barycenter weights sum to 1. Parameters ---------- X : array-like, shape (n_samples, n_dim) Y : array-like, shape (n_samples, n_dim) indices : array-like, shape (n_samples, n_dim) Indices of the points in Y used to compute the barycenter reg : float, default=1e-3 Amount of regularization to add for the problem to be well-posed in the case of n_neighbors > n_dim Returns ------- B : array-like, shape (n_samples, n_neighbors) Notes ----- See developers note for more information. dtyperNpos)Zassume_a)rrintshapeAssertionErrornpemptyrones enumeratedotTtraceflatrsum)XYindicesreg n_samples n_neighborsBviindACGr'Rwr9D/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/manifold/_locally_linear.pybarycenter_weightss&       r;c Cst|d|d|}|j}|j}|j|ddddddf}t||||d}td||d|}t| | |f||fdS) a-Computes the barycenter weighted graph of k-Neighbors for points in X Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : int Number of neighbors for each sample. reg : float, default=1e-3 Amount of regularization when solving the least-squares problem. Only relevant if mode='barycenter'. If None, use the default. n_jobs : int or None, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- A : sparse matrix in CSR format, shape = [n_samples, n_samples] A[i, j] is assigned the weight of edge that connects i to j. See Also -------- sklearn.neighbors.kneighbors_graph sklearn.neighbors.radius_neighbors_graph rr/n_jobsF)return_distanceNr-r)r) rfit_fit_XZn_samples_fit_ kneighborsr;r!ZarangerZravel) r*r/r-r=Zknnr.r3dataZindptrr9r9r:barycenter_kneighbors_graphTs!rDrarpackư>dc Cs0|dkr,|jddkr(||dkr(d}nd}|dkrt|jd|}z t|||d|||d\}} Wn0tk r} ztd | | W5d } ~ XYnX| d d |d ft||d fS|dkr t|d r|}t ||||d fd d\}} t t |} | d d | ft|fStd|d S)a0 Find the null space of a matrix M. Parameters ---------- M : {array, matrix, sparse matrix, LinearOperator} Input covariance matrix: should be symmetric positive semi-definite k : int Number of eigenvalues/vectors to return k_skip : int, default=1 Number of low eigenvalues to skip. eigen_solver : {'auto', 'arpack', 'dense'}, default='arpack' auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, default=1e-6 Tolerance for 'arpack' method. Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for 'arpack' method. Not used if eigen_solver=='dense' random_state : int, RandomState instance, default=None Determines the random number generator when ``solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. autor rEdenseg)sigmatolmaxiterv0a Error in determining null-space with ARPACK. Error message: '%s'. Note that eigen_solver='arpack' can fail when the weight matrix is singular or otherwise ill-behaved. In that case, eigen_solver='dense' is recommended. See online documentation for more information.NtoarrayrT)Zsubset_by_indexZ overwrite_azUnrecognized eigen_solver '%s') rrr RuntimeError ValueErrorr!r)hasattrrPrZargsortabs) Mkk_skip eigen_solverrMmax_iter random_staterOZ eigen_valuesZ eigen_vectorseindexr9r9r: null_space~sF*&   r]rHstandard-C6?-q=) r-rXrMrYmethod hessian_tol modified_tolrZr=c ; Cs|dkrtd||dkr(td|t|d| d} | || j}|j\} }||krbtd|| krztd| |f|d krtd |d k}|d krt| ||| d }|rt|jd|ji|}|j| }n:|j||j| }|j dd|jd dd7<n|dkrH||dd}|||krDtd| j ||ddd}|ddddf}t j|d||ft jd}d|ddd f<t j| | ft jd}||k}t| D]v}|||}||d 8}|rt|d dd }n,t ||j}t|ddddddf}|ddd|f|dddd|f<d|}t|D]V}|dd||df|dd||f|dd||||f<|||7}qXt|\}}|dd|ddf}|d }d|t t||k<||}t ||||\} }!|| |!ft ||j7<q|rt|}n|dkrv||krdtd| j ||ddd}|ddddf}t | ||f}"t||}#t | |#g}$||k}|r t| D]4}|||||}%t|%dd\|"|<|$|<}&q|$dC}$njt| D]`}|||||}%t |%|%j}'t|'\}(})|(ddd|$|<|)dddddf|"|<qd|$d}t |"d ddt |}*|*ddd|#f|$|dddf<|*dd|#df|dddf<t | |f}+t| D]}t |"||*||+|<q|+|+ddddf}+|$dd|dfd|$ddd|fd},t |,}-t j| t d}.t!|$d}/|/ddddf|/ddddfd}0t| D]$}t "|0|dddf|-|.|<q|.||#7}.t j| | ft jd}t| D]F}|.|}1|"|dd||1df}2t j#$|2d t %|1}3t &|1|3t |2jt |}4t j#$|4}5|5| kr|4d 9}4n|4|5}4|2dt 't |2|4|4d|3|+|dddf}6t ||||\} }!|| |!ft |6|6j7<|6d}7||||f|78<||||f|78<|||f|17<q|rt|}n^|dkr| j ||ddd}|ddddf}t | | f}||k}t| D] }|||}8|8|8d 8}8|rt|8ddd }9n,t |8|8j}t|ddddddf}9t ||df}|9ddd|f|ddddf<dt %||ddd f<t ||j}:t ||||\} }!|| |!f|:8<|||||fd7<qt(||d|||| dS)acPerform a Locally Linear Embedding analysis on the data. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : int Number of neighbors to consider for each point. n_components : int Number of coordinates for the manifold. reg : float, default=1e-3 Regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : {'auto', 'arpack', 'dense'}, default='auto' auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, default=1e-6 Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for the arpack solver. method : {'standard', 'hessian', 'modified', 'ltsa'}, default='standard' standard : use the standard locally linear embedding algorithm. see reference [1]_ hessian : use the Hessian eigenmap method. This method requires n_neighbors > n_components * (1 + (n_components + 1) / 2. see reference [2]_ modified : use the modified locally linear embedding algorithm. see reference [3]_ ltsa : use local tangent space alignment algorithm see reference [4]_ hessian_tol : float, default=1e-4 Tolerance for Hessian eigenmapping method. Only used if method == 'hessian'. modified_tol : float, default=1e-12 Tolerance for modified LLE method. Only used if method == 'modified'. random_state : int, RandomState instance, default=None Determines the random number generator when ``solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int or None, default=None The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- Y : array-like, shape [n_samples, n_components] Embedding vectors. squared_error : float Reconstruction error for the embedding vectors. Equivalent to ``norm(Y - W Y, 'fro')**2``, where W are the reconstruction weights. References ---------- .. [1] Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000). .. [2] Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003). .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. `_ .. [4] Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004) )rHrErKzunrecognized eigen_solver '%s')r^hessianmodifiedltsazunrecognized method '%s'rr<z>output dimension must be less than or equal to input dimensionzHExpected n_neighbors <= n_samples, but n_samples = %d, n_neighbors = %drzn_neighbors must be positiverKr^)r/r-r=formatNrdr z^for method='hessian', n_neighbors must be greater than [n_components * (n_components + 3) / 2]Fr/r>r)Z full_matricesrez1modified LLE requires n_neighbors >= n_componentsTrrfg?)rWrXrMrYrZ))rRrr@rArrDrrgr&ZtocsrrPr(rBr!r"Zfloat64zerosrangeZmeanrr%rrr)whererTZmeshgridrminZ transposer#ZmedianrrZ searchsortedZlinalgZnormsqrtfullouterr]);r*r/ n_componentsr-rXrMrYrarbrcrZr=ZnbrsNZd_inZM_sparseWrUZdp neighborsZYiZuse_svdr2ZGiUZCijrVQr7r8SZnbrs_xZnbrs_yVZnevZevalsZX_nbrs_ZC_nbrsZevivitmpZw_regrhoetaZs_rangeZ evals_cumsumZ eta_rangeZs_iZViZalpha_ihZnorm_hZWiZWi_sum1Xir1ZGiGiTr9r9r:locally_linear_embeddingsBn     &  ( D  "        ,( 4  , "     6    $ rc@seZdZUdZeeddddgeeddddgeeddddgeddd hgeeddddgeeddddged d d d hgeeddddgeeddddgeddddhgdgdegd Ze e d<ddddddd dddddd ddZ ddZ d%dd Z d&d!d"Zd#d$ZdS)'LocallyLinearEmbeddingaLocally Linear Embedding. Read more in the :ref:`User Guide `. Parameters ---------- n_neighbors : int, default=5 Number of neighbors to consider for each point. n_components : int, default=2 Number of coordinates for the manifold. reg : float, default=1e-3 Regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : {'auto', 'arpack', 'dense'}, default='auto' The solver used to compute the eigenvectors. The available options are: - `'auto'` : algorithm will attempt to choose the best method for input data. - `'arpack'` : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. - `'dense'` : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. .. warning:: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. tol : float, default=1e-6 Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : int, default=100 Maximum number of iterations for the arpack solver. Not used if eigen_solver=='dense'. method : {'standard', 'hessian', 'modified', 'ltsa'}, default='standard' - `standard`: use the standard locally linear embedding algorithm. see reference [1]_ - `hessian`: use the Hessian eigenmap method. This method requires ``n_neighbors > n_components * (1 + (n_components + 1) / 2``. see reference [2]_ - `modified`: use the modified locally linear embedding algorithm. see reference [3]_ - `ltsa`: use local tangent space alignment algorithm. see reference [4]_ hessian_tol : float, default=1e-4 Tolerance for Hessian eigenmapping method. Only used if ``method == 'hessian'``. modified_tol : float, default=1e-12 Tolerance for modified LLE method. Only used if ``method == 'modified'``. neighbors_algorithm : {'auto', 'brute', 'kd_tree', 'ball_tree'}, default='auto' Algorithm to use for nearest neighbors search, passed to :class:`~sklearn.neighbors.NearestNeighbors` instance. random_state : int, RandomState instance, default=None Determines the random number generator when ``eigen_solver`` == 'arpack'. Pass an int for reproducible results across multiple function calls. See :term:`Glossary `. n_jobs : int or None, default=None The number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- embedding_ : array-like, shape [n_samples, n_components] Stores the embedding vectors reconstruction_error_ : float Reconstruction error associated with `embedding_` n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 nbrs_ : NearestNeighbors object Stores nearest neighbors instance, including BallTree or KDtree if applicable. See Also -------- SpectralEmbedding : Spectral embedding for non-linear dimensionality reduction. TSNE : Distributed Stochastic Neighbor Embedding. References ---------- .. [1] Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000). .. [2] Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003). .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. `_ .. [4] Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004) Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.manifold import LocallyLinearEmbedding >>> X, _ = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> embedding = LocallyLinearEmbedding(n_components=2) >>> X_transformed = embedding.fit_transform(X[:100]) >>> X_transformed.shape (100, 2) rNleft)closedrrHrErKr^rdrerfZbruteZkd_treeZ ball_treerZ) r/rqr-rXrMrYrarbrcneighbors_algorithmrZr=_parameter_constraintsr rrFrGr_r`c CsL||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ dS)N) r/rqr-rXrMrYrarbrcrZrr=) selfr/rqr-rXrMrYrarbrcrrZr=r9r9r:__init__szLocallyLinearEmbedding.__init__cCst|j|j|jd|_t|j}|j|td}|j |t |j|j|j |j |j |j|j|j|j||j|jd \|_|_|jjd|_dS)N)r/ algorithmr=r) r*r/rqrXrMrYrarbrcrZr-r=r)rr/rr=nbrs_rrZ_validate_datafloatr@rrqrXrMrYrarbrcr- embedding_Zreconstruction_error_rZ_n_features_out)rr*rZr9r9r:_fit_transforms.  z%LocallyLinearEmbedding._fit_transformcCs||||S)ayCompute the embedding vectors for data X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. y : Ignored Not used, present here for API consistency by convention. Returns ------- self : object Fitted `LocallyLinearEmbedding` class instance. )_validate_paramsrrr*yr9r9r:r@s zLocallyLinearEmbedding.fitcCs||||jS)aCompute the embedding vectors for data X and transform X. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. y : Ignored Not used, present here for API consistency by convention. Returns ------- X_new : array-like, shape (n_samples, n_components) Returns the instance itself. )rrrrr9r9r: fit_transforms z$LocallyLinearEmbedding.fit_transformcCst||j|dd}|jj||jdd}t||jj||jd}t |j d|j f}t |j dD]$}t |j||j||||<qd|S)a Transform new points into embedding space. Parameters ---------- X : array-like of shape (n_samples, n_features) Training set. Returns ------- X_new : ndarray of shape (n_samples, n_components) Returns the instance itself. Notes ----- Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs). F)resetrhr?r)rrrrBr/r;rAr-r!r"rrqrkr%rr&)rr*r3weightsZX_newr2r9r9r: transform"s"z LocallyLinearEmbedding.transform)N)N)__name__ __module__ __qualname____doc__rrrrrdict__annotations__rrr@rrr9r9r9r:r*s@    r)r)rN)rrErFrGN)*rZnumbersrrZnumpyr!Z scipy.linalgrrrZ scipy.sparserrZscipy.sparse.linalgr baser r r rutilsrrZ utils._arpackrZutils._param_validationrrZ utils.fixesrZ utils.extmathrZutils.validationrrrtrr;rDr]rrr9r9r9r:sP        6 + Q b