U 3dL@sddlZddlmZddlZddlZddlmZm Z m Z ddl m Z m Z ddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZd dZddZ ddZ!ddZ"ddZ#ej$%dddddgej$%dddgdd Z&d!d"Z'd#d$Z(d%d&Z)d'd(Z*ej$%dddddgd)d*Z+d+d,Z,d-d.Z-d/d0Z.d1d2Z/ej$%dddddgd3d4Z0ej$%d5ddd6gd7d8Z1d9d:Z2d;d<Z3d=d>Z4d?d@Z5dS)AN)assert_array_almost_equalassert_array_equalassert_allclose)PCA KernelPCA) make_circles) make_blobs)NotFittedError) Perceptron)Pipeline)StandardScaler) GridSearchCV) rbf_kernel)_check_psd_eigenvaluesc Cstjd}|d}|d}dd}dD]}ddd |fD]}t| }td |||d }||}|||} t t |t | |j dkst ||} | j d |j d kst |r<|| } | j |j ks.histogram)autodensearpack randomizedlinearrbfpolyr)kernel eigen_solverfit_inverse_transformN)rrandom RandomState random_samplecallabler fit_transformfit transformrabssizershapeinverse_transform) rngX_fitX_predrr&r%invkpcaX_fit_transformedX_fit_transformed2X_pred_transformedZX_pred2rrrtest_kernel_pcas2       r<c CsDtdddd}d}tjt|d|tjddW5QRXdS)zCheck that kPCA raises an error if the parameters are invalid Tests fitting inverse transform with a precomputed kernel raises a ValueError. T precomputed) n_componentsr'r%z6Cannot fit_inverse_transform with a precomputed kernelmatchN)rpytestraises ValueErrorr.rr)randn)Z estimatorZerr_msrrr"test_kernel_pca_invalid_parametersFsrFcCsbtjd}|dd}t|d|}||}|}d|dddf<||}t||dS)zCheck robustness to mutations in the original training array Test that after fitting a kPCA model, it stays independent of any mutation of the values of the original data object by relying on an internal copy. rr=) random_stateiN) rr)r*randrr.r/copyr)stateXr8Z transformed1ZX_copyZ transformed2rrr$test_kernel_pca_consistent_transformTs    rLc Cstjd}|dd}d}|D]j}td}tdD],}td||d}||d||ddf<q6t|t |dddfd ddq dS) zTest that Kernel PCA produces deterministic output Tests that the same inputs and random state produce the same output. rr=)r r)rrMr)r?r&rGN) rr)r*rHzerosrangerr-rZtilereshape)r4rKr&solverZ transformed_Xir8rrr$test_kernel_pca_deterministic_outputgs    rSc Cstjd}t|d}t|d}dD]}dD]}td||ddd}||}|| |}t t |t || |}|j d |j d kst tt||W5QRXq8q0d S) zTest that kPCA works on a sparse data input. Same test as ``test_kernel_pca except inverse_transform`` since it's not implemented for sparse matrices. rrr)rr r!)r"r#r$rF)r%r&r'rGr(N)rr)r*spZ csr_matrixr+rr-r.r/rr0r2rrBrCr r3) r4r5r6r&r%r8r9r:r;rrrtest_kernel_pca_sparsexs,    rUrQrrr r! n_featuresrr=cCstjd}|d|f}|d|f}|dkr4dnd}ttt||d||tt ||dkrj|nd d ||d S) zTest that kPCA with linear kernel is equivalent to PCA for all solvers. KernelPCA with linear kernel should produce the same output as PCA. rrrr r)r&rfull)Z svd_solverN) rr)r*r+rr0rr.r/r)rQrVr4r5r6Zn_compsrrrtest_kernel_pca_linear_kernels rYcCsftjd}|d}|d}dD]<}dD]2}t||d}|||j}|d|fks,tq,q$dS) zTest that `n_components` is correctly taken into account for projections For all solvers this tests that the output has the correct shape depending on the selected number of components. rrrrr r!)r(rr)r?r&rN) rr)r*r+rr.r/r2r)r4r5r6r&cr8r2rrrtest_kernel_pca_n_componentss    r\cCstddgddgddgg}t}||}|jdks:ttdd}||}|jdks\ttddd}||}|jdkstd S) zCheck that the ``remove_zero_eig`` parameter works correctly. Tests that the null-space (Zero) eigenvalues are removed when remove_zero_eig=True, whereas they are not by default. ?r()rWrrr?)rWrT)r?remove_zero_eigN)rarrayrr-r2r)rKr8ZXtrrrtest_remove_zero_eigs     rac Cstddgddgg}thtdttjddDtdddd }|| |}| |}t t |t |W5QRXW5QRXd S) zNon-regression test for issue #12141 (PR #12143) This test checks that fit().transform() returns the same result as fit_transform() in case of non-removed zero eigenvalue. r(rerrorwarn)allrFr)r?r_r&N) rr`warningscatch_warnings simplefilterRuntimeWarningZerrstaterr.r/r-rr0)r5kABrrrtest_leave_zero_eigs   rlcCstjd}|d}|d}dD]}td|dd||}td|dddt||jt||j}td|ddd t||j}td|dddt||jt||j}t t |t |t t |t |q$d S) z?Test that kPCA works with a precomputed kernel, for all solversrrrrZrr&rGr>)r&r%rGN) rr)r*r+rr.r/dotTr-rr0)r4r5r6r&X_kpcaZX_kpca2Z X_kpca_trainZ X_kpca_train2rrrtest_kernel_pca_precomputedsT        rqcCstddgddgg}td|ddd}||dd gd dgg}td|ddd}||t|j|jt|j|jd S) zCheck that the kernel centerer works. Tests that a non symmetric precomputed kernel is actually accepted because the kernel centerer does its job correctly. r(rrW(r>r)r%r&r?rG iN)rr.rZ eigenvectors_ eigenvalues_)rQKr8ZKcZkpca_crrr)test_kernel_pca_precomputed_non_symmetrics$   rvcCs|tddddd\}}tddd}td |fd td d fg}td tddd}t|d|d}||||j dksxt dS)zCheck that kPCA works as expected in a grid search pipeline Test if we can do a grid-search to find parameters to separate circles with a perceptron model. 333333?皙?r n_samplesZfactorZnoiserGr#rr%r? kernel_pcar rZmax_iter@)Zkernel_pca__gammarWZcv param_gridr(N) rrr r dictraranger r. best_score_r)rKrr8pipeliner grid_searchrrrtest_gridsearch_pipeline9s  rcCstddddd\}}tddd}td |fd td d fg}ttd d d}t|d|d}t|dd}| |||j d kst dS)zCheck that kPCA works as expected in a grid search pipeline (2) Test if we can do a grid-search to find parameters to separate circles with a perceptron model. This test uses a precomputed kernel. rwrxryrrzr>rr|r}r rr~r()ZPerceptron__max_iterrWrr)gammaN) rrr r rrrr rr.rr)rKrr8rrrZX_kernelrrr$test_gridsearch_pipeline_precomputedHs   rcCs~tddddd\}}tdd||||}|dks:ttd d d d d }||}tdd||||}|dksztdS)aCheck that kPCA projects in a space where nested circles are separable Tests that 2D nested circles become separable with a perceptron when projected in the first 2 kPCA using an RBF kernel, while raw samples are not directly separable in the original space. rwrxryrrzrr~g?r#rTr)r%r?r'rr]N)rr r.Zscorerrr-)rKrZ train_scorer8rprrrtest_nested_circlesXs  rcCs^ddgddgddgg}tdddd }|||jdks@tt|jt|jksZtd S) z}Check that ``_check_psd_eigenvalues`` is correctly called in kPCA Non-regression test for issue #12140 (PR #12145). rr(gw̫@g:0yE>rr"rT)r%r?r'N)rr.rtminrrrdrrKr8rrrtest_kernel_conditioningts  rcCsddddddddgdddd d d d d gddd d d d ddgdd d dddddgdd d dddddgdd d dddddgdd ddddddgdd ddddddgg}td|dd}tjtdd||W5QRXtd|dd}|dkrtjtdd||W5QRXn ||dS)aCheck how KernelPCA works with non-PSD kernels depending on n_components Tests for all methods what happens with a non PSD gram matrix (this can happen in an isomap scenario, or with custom kernel functions, or maybe with ill-posed datasets). When ``n_component`` is large enough to capture a negative eigenvalue, an error should be raised. Otherwise, KernelPCA should run without error since the negative eigenvalues are not selected. gQ@ggp= # @gp= ף@g ףp= g(\)gQg@gףp= gQg(\@g(\.@gQ@g333333&g{G:7g@g333333 gRQ?ggq= ףp@g#@g(\u5@r>)r%r&r?z*There are significant negative eigenvaluesr@rr!N)rrBrCrDr.)rQrur8rrrtest_precomputed_kernel_not_psds( rr?rMc Csd\}}t||dddd\}}|d|ddf||dddf}}t|ddd||}t|d dd||}tt|t|t|d dd||} tt| t|dS) zGCheck that 'dense' 'arpack' & 'randomized' solvers give similar results)idrxryrrzNrrmr r!)rrr.r/rrr0) r?Zn_trainZn_testrK_r5r6Zref_predZa_predZr_predrrr#test_kernel_pca_solvers_equivalences8 *rcCs^tdddd^}}tddddd }||}||}tj||tj|d ksZtd S) zTest if the reconstruction is a good approximation. Note that in general it is not possible to get an arbitrarily good reconstruction because of kernel centering that does not preserve all the information of the original data. rrrr{rVrGrMr#TgMbP?)r?r%r'alpha皙?N)rrr-r3rZlinalgZnormr)rKrr8ZX_transZ X_reconstrrr0test_kernel_pca_inverse_transform_reconstructions  rc CsHtjddd}t}||tt| |W5QRXdS)NrrW) rr)rErPrr.rBrCr r3rrrr&test_kernel_pca_raise_not_fitted_errors   rcCsjtddddgdddggddd\}}t|}||8}t}||j||tjjksft dS)zTest that the decomposition is similar for 32 and 64 bits data Non regression test for https://github.com/scikit-learn/scikit-learn/issues/18146 rr(r)r{ZcentersrGZ cluster_stdN) rr r-rrr2ZastyperZfloat32r)rKrr8rrrtest_32_64_decomposition_shapes   rcCsFtdddd^}}tdd|}|}tddtdD|d S) z&Check feature names out for KernelPCA.rrrrrr^cSsg|]}d|qS)Z kernelpcar).0rRrrr sz5test_kernel_pca_feature_names_out..N)rrr.Zget_feature_names_outrrO)rKrr8namesrrr!test_kernel_pca_feature_names_outsr)6ZnumpyrZ scipy.sparsesparserTrBreZsklearn.utils._testingrrrZsklearn.decompositionrrZsklearn.datasetsrrZsklearn.exceptionsr Zsklearn.linear_modelr Zsklearn.pipeliner Zsklearn.preprocessingr Zsklearn.model_selectionr Zsklearn.metrics.pairwiserZsklearn.utils.validationrr<rFrLrSrUmarkZ parametrizerYr\rarlrqrvrrrrrrrrrrrrrrsP          .% &  3 $