U 3dړ @sdZddlZddlZddlZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZdgZdd Zd d Zd d ZddZdddZddZddZdddZdS)a\ scikit-learn copy of scipy/sparse/linalg/_eigen/lobpcg/lobpcg.py v1.10 to be deleted after scipy 1.4 becomes a dependency in scikit-lean ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Locally Optimal Block Preconditioned Conjugate Gradient Method (LOBPCG). References ---------- .. [1] A. V. Knyazev (2001), Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. :doi:`10.1137/S1064827500366124` .. [2] A. V. Knyazev, I. Lashuk, M. E. Argentati, and E. Ovchinnikov (2007), Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc. :arxiv:`0705.2626` .. [3] A. V. Knyazev's C and MATLAB implementations: https://github.com/lobpcg/blopex N)inveigh cho_factor cho_solvecholesky LinAlgError)LinearOperator) isspmatrix)blocklobpcgc Csddlm}||j}||d}dt|jj}t||||d}||kr~t j d|d|jd|d|d t d d d S) zA Report if `M` is not a Hermitian matrix given its type. r)norm zMatrix z of the type z is not Hermitian: condition: z < z fails. stacklevelN) scipy.linalgr Tconjnpfinfodtypeepsmaxwarningswarn UserWarning)Mnamer ZmdZnmdtolr =/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/externals/_lobpcg.py_report_nonhermitian"s  r"cCs4|jdkr|Stj|dd}|jddf|_|SdS)zm If the input array is 2D return it, if it is 1D, append a dimension, making it a column vector. F)copyrr N)ndimrarrayshape)arauxr r r!_as2d3s  r*cs0dkr dStr fddSfddSdS)Ncs|SNr vmr r!Dz_makeMatMat..cs|Sr+r r,r.r r!r0Fr1)callabler.r r.r! _makeMatMat@s  r3cCs0t|j|}t||}|t||8}dS)zChanges blockVectorV in place.N)rdotrrr) blockVectorVZfactYBY blockVectorBY blockVectorYZYBVtmpr r r!_applyConstraintsIs r9c CsT|jddt|jj}||}|dkr|dk rz ||}WnRtk r}z4|rtjd|dtddddd|fWYSW5d}~XYnX|j |j krt d|j d |j d q|}n||}|j |}z@t |d d }t|d d }||}|dk r ||}||||fWStk rN|r>tjd tddddd|fYSXdS)z@in-place B-orthonormalize the given block vector using Cholesky.r)ZaxisN(Secondary MatMul call failed with error  r The shape z; of the orthogonalized matrix not preserved and changed to , after multiplying by the secondary matrix. T)Z overwrite_azCholesky has failed.)rrrrr Exceptionrrrr' ValueErrorrrrrr)Br5Z blockVectorBVverbosityLevelZ normalizationeZVBVr r r!_b_orthonormalizePsH  &    rDcCs4t|}|r$|d| dd}n |d|}|S)z?Get `num` indices into `_lambda` depending on `largest` option.Nr )rZargsort)_lambdanumlargestiir r r! _get_indxs   rJcCs|rt|dt|ddS)NgramAgramB)r")rKrLrBr r r!_handle_gramA_gramB_verbositys rMTFc O CsT|} | } |}|}|dkrd}|}d}|dk rht|jdkr^tjdt|jdtddd}n |jd}| dkrxtd t| jdkrtd | j\}}t| jtj stjd | jd tddtj | tj d } | rtj |d|f| jd }| r tj |d|f| jd }|rd}|dkr*|d7}n|d7}|d7}|dkrL|d7}|d7}|d|7}|d|7}|dkr|d7}n$|dkr|d|7}n |d|7}t |||d|krDtjd|d|d|dtddt||}|dk rtd|r|||df}n d|df}zt|trD|tj|td }n^t|r|tj|td }|j||fkrtd |jd!nt|r|}n t |}Wn4tk r}ztd"|d#W5d}~XYnX|dk rzt|tr|tj|td }n^t|rF|tj|td }|j||fkrdtd |jd$nt|rZ|}n t |}Wn4tk r}ztd%|d#W5d}~XYnXzrd&ttjkrd&|i}nd'|i}t||fd(d)i|\}}|r|ddd*}|ddddd*f}||fWStk rB}ztd+|d#W5d}~XYnX|dksX|d,krptt| jj|}t|}t|}t|}|dk r.|dk r||}|j|jkrtd |jd-|jd.n|}t |j!"|}z t#|}Wn.t$k r}ztd/|W5d}~XYnXt%| |||t&|| |d0\} }}}| dkrVtd1|| } | j| jkrtd | jd2| jd3t | j!"| }!t|!d)d4\}"}#t'|"||}$|"|$}"| r|"|dddf<t |#dd|$f}#t | |#} t | |#} |dk rt ||#}tj(|ft)d }%d}&d}'d}(t*t| jj+})d*}*d5}+d)},d)}-|*|kr|*d7}*|dk r||"tj,ddf}n| |"tj,ddf}| |}.t-|."|.d}tt*|}/| r|/||*ddf<t-t*|/|}0|0|)kr|0})|*}| } n|0d| |)krd5},|| } | j| jkrTtd | jd6| jd3|dk r|| }|j| jkrtd | jd6|jd.t.|/|kd5d)}$|%|$@}%|%-}1|rt d7|*t d8|1t d9|"t d:|/|1dkrqt/|.dd|%f}2|*dkr`t/|&dd|%f}3t/|'dd|%f}4|dk r`t/|(dd|%f}5|dk rr||2}2|dk rt%|2||||dk r|2| |j!"|2}2n|2| | j!"|2}2t&||2|d0}|\}2}6}}|2dk rtjd;|*d<|/d=|d>tddq||2}7|*dk r|dk rHt&||3|5|d0}|\}3}5}8}9nt&||3|d0}|\}3}}8}9|3dk r|4|9}4t |4|8}4|,}+nd5}+|7jd?k rd}:ntt|2jj}:|/+|:k r|- sd)}-nd5}-|dk r| }|2}6|+ s|3}5t | j!"|7};t |2j!"|7}<|7j}=|- r|<|t |2j!"|6}?t | j!"|6}@n>t0|"1|=}!tj||=d }>tj|1|=d }?tj ||1f|=d }@|+ s4t | j!"|4}At |2j!"|4}Bt |3j!"|4}Ct | j!"|5}Dt |2j!"|5}E|- rR|C|Cj!"d}Ct |3j!"|5}Fntj|1|=d }Ft2|!|;|Ag|;j!"|<|Bg|Aj!"|Bj!"|Cgg}Gt2|>|@|Dg|@j!"|?|Eg|Dj!"|Ej!"|Fgg}Ht3|G|H|zt|G|Hd)d4\}"}#WnLt$k r2}z,| rtjd@|*dA|dBtddd5}+W5d}~XYnX|+ rt2|!|;g|;j!"||@g|@j!"|?gg}Ht3|G|H|zt|G|Hd)d4\}"}#WnLt$k r}z,tjd@|*dC|d#tddWYqW5d}~XYnXt'|"||}$|"|$}"|#dd|$f}#| r|"||*dddf<|dk r,|+ s|#d|}I|#|||1}J|#||1d}Kt |2|J}L|Lt |3|K7}Lt |7|J}M|Mt |4|K7}Mt |6|J}N|Nt |5|K7}Nn<|#d|}I|#|d}Jt |2|J}Lt |7|J}Mt |6|J}Nt | |I|L} t | |I|M} t ||I|N}|L|M|N}&}'}(n|+s|#d|}I|#|||1}J|#||1d}Kt |2|J}L|Lt |3|K7}Lt |7|J}M|Mt |4|K7}Mn0|#d|}I|#|d}Jt |2|J}Lt |7|J}Mt | |I|L} t | |I|M} |L|M}&}'qV|dk r||"tj,ddf}n| |"tj,ddf}| |}.t-|."|.d}tt*|}/| rt|"||*dddf<| r|/||*dddf<t-t*|/|}0|0|)kr|0})|*d}| } t+t*|/|krtjdD|*dE|/dF|dG|dH|)dI tdd|r$t dJ|"t dK|/| } |dk r@t%| ||||| } | j| jkrptd | jdL| jd3t | j!"| }!| }|dk r|| }|j| jkrtd | jdL|jd.t | j!"|}>t3|!|>||!|!j!"d}!|>|>j!"d}>zt|!|>d)d4\}"}#Wn.t$k rF}ztdM|W5d}~XYnXt'|"||}$|"|$}"t |#dd|$f}#t | |#} t | |#} |dk rt ||#}||"tj,ddf}n| |"tj,ddf}| |}.t-|."|.d}tt*|}/| r|"||dddf<| r.|/||dddf<| rL|d|dddf}| rj|d|dddf}t+t*|/|krtjdN|/dF|d>tdd|rt dO|"t dP|/| rt4|t|d}dQdR|D}| rt4|t|d}dSdR|D}| r8| r,|"| ||fS|"| |fSn| rH|"| |fS|"| fSdS)TaLocally Optimal Block Preconditioned Conjugate Gradient Method (LOBPCG). LOBPCG is a preconditioned eigensolver for large symmetric positive definite (SPD) generalized eigenproblems. Parameters ---------- A : {sparse matrix, dense matrix, LinearOperator, callable object} The symmetric linear operator of the problem, usually a sparse matrix. Often called the "stiffness matrix". X : ndarray, float32 or float64 Initial approximation to the ``k`` eigenvectors (non-sparse). If `A` has ``shape=(n,n)`` then `X` should have shape ``shape=(n,k)``. B : {dense matrix, sparse matrix, LinearOperator, callable object} Optional. The right hand side operator in a generalized eigenproblem. By default, ``B = Identity``. Often called the "mass matrix". M : {dense matrix, sparse matrix, LinearOperator, callable object} Optional. Preconditioner to `A`; by default ``M = Identity``. `M` should approximate the inverse of `A`. Y : ndarray, float32 or float64, optional. An n-by-sizeY matrix of constraints (non-sparse), sizeY < n. The iterations will be performed in the B-orthogonal complement of the column-space of Y. Y must be full rank. tol : scalar, optional. Solver tolerance (stopping criterion). The default is ``tol=n*sqrt(eps)``. maxiter : int, optional. Maximum number of iterations. The default is ``maxiter=20``. largest : bool, optional. When True, solve for the largest eigenvalues, otherwise the smallest. verbosityLevel : int, optional Controls solver output. The default is ``verbosityLevel=0``. retLambdaHistory : bool, optional. Whether to return eigenvalue history. Default is False. retResidualNormsHistory : bool, optional. Whether to return history of residual norms. Default is False. restartControl : int, optional. Iterations restart if the residuals jump up 2**restartControl times compared to the smallest ones recorded in retResidualNormsHistory. The default is ``restartControl=20``, making the restarts rare for backward compatibility. Returns ------- w : ndarray Array of ``k`` eigenvalues. v : ndarray An array of ``k`` eigenvectors. `v` has the same shape as `X`. lambdas : ndarray, optional The eigenvalue history, if `retLambdaHistory` is True. rnorms : ndarray, optional The history of residual norms, if `retResidualNormsHistory` is True. Notes ----- The iterative loop in lobpcg runs maxit=maxiter (or 20 if maxit=None) iterations at most and finishes earler if the tolerance is met. Breaking backward compatibility with the previous version, lobpcg now returns the block of iterative vectors with the best accuracy rather than the last one iterated, as a cure for possible divergence. The size of the iteration history output equals to the number of the best (limited by maxit) iterations plus 3 (initial, final, and postprocessing). If both ``retLambdaHistory`` and ``retResidualNormsHistory`` are True, the return tuple has the following format ``(lambda, V, lambda history, residual norms history)``. In the following ``n`` denotes the matrix size and ``k`` the number of required eigenvalues (smallest or largest). The LOBPCG code internally solves eigenproblems of the size ``3k`` on every iteration by calling the dense eigensolver `eigh`, so if ``k`` is not small enough compared to ``n``, it makes no sense to call the LOBPCG code. Moreover, if one calls the LOBPCG algorithm for ``5k > n``, it would likely break internally, so the code calls the standard function `eigh` instead. It is not that ``n`` should be large for the LOBPCG to work, but rather the ratio ``n / k`` should be large. It you call LOBPCG with ``k=1`` and ``n=10``, it works though ``n`` is small. The method is intended for extremely large ``n / k``. The convergence speed depends basically on two factors: 1. Relative separation of the seeking eigenvalues from the rest of the eigenvalues. One can vary ``k`` to improve the absolute separation and use proper preconditioning to shrink the spectral spread. For example, a rod vibration test problem (under tests directory) is ill-conditioned for large ``n``, so convergence will be slow, unless efficient preconditioning is used. For this specific problem, a good simple preconditioner function would be a linear solve for `A`, which is easy to code since `A` is tridiagonal. 2. Quality of the initial approximations `X` to the seeking eigenvectors. Randomly distributed around the origin vectors work well if no better choice is known. References ---------- .. [1] A. V. Knyazev (2001), Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method. SIAM Journal on Scientific Computing 23, no. 2, pp. 517-541. :doi:`10.1137/S1064827500366124` .. [2] A. V. Knyazev, I. Lashuk, M. E. Argentati, and E. Ovchinnikov (2007), Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc. :arxiv:`0705.2626` .. [3] A. V. Knyazev's C and MATLAB implementations: https://github.com/lobpcg/blopex Examples -------- Solve ``A x = lambda x`` with constraints and preconditioning. >>> import numpy as np >>> from scipy.sparse import spdiags, issparse >>> from scipy.sparse.linalg import lobpcg, LinearOperator The square matrix size: >>> n = 100 >>> vals = np.arange(1, n + 1) The first mandatory input parameter, in this test a sparse 2D array representing the square matrix of the eigenvalue problem to solve: >>> A = spdiags(vals, 0, n, n) >>> A.toarray() array([[ 1, 0, 0, ..., 0, 0, 0], [ 0, 2, 0, ..., 0, 0, 0], [ 0, 0, 3, ..., 0, 0, 0], ..., [ 0, 0, 0, ..., 98, 0, 0], [ 0, 0, 0, ..., 0, 99, 0], [ 0, 0, 0, ..., 0, 0, 100]]) Initial guess for eigenvectors, should have linearly independent columns. The second mandatory input parameter, a 2D array with the row dimension determining the number of requested eigenvalues. If no initial approximations available, randomly oriented vectors commonly work best, e.g., with components normally disrtibuted around zero or uniformly distributed on the interval [-1 1]. >>> rng = np.random.default_rng() >>> X = rng.normal(size=(n, 3)) Constraints - an optional input parameter is a 2D array comprising of column vectors that the eigenvectors must be orthogonal to: >>> Y = np.eye(n, 3) Preconditioner in the inverse of A in this example: >>> invA = spdiags([1./vals], 0, n, n) The preconditiner must be defined by a function: >>> def precond( x ): ... return invA @ x The argument x of the preconditioner function is a matrix inside `lobpcg`, thus the use of matrix-matrix product ``@``. The preconditioner function is passed to lobpcg as a `LinearOperator`: >>> M = LinearOperator(matvec=precond, matmat=precond, ... shape=(n, n), dtype=np.float64) Let us now solve the eigenvalue problem for the matrix A: >>> eigenvalues, _ = lobpcg(A, X, Y=Y, M=M, largest=False) >>> eigenvalues array([4., 5., 6.]) Note that the vectors passed in Y are the eigenvectors of the 3 smallest eigenvalues. The results returned are orthogonal to those. NrNrr#z2Expected rank-2 array for argument Y, instead got z&, so ignore it and use no constraints.rr z-The mandatory initial matrix X cannot be Nonez$expected rank-2 array for argument XzData type for argument X is z0, which is not inexact, so casted to np.float32.)rr<zSolving standardZ generalizedz eigenvalue problem withoutz preconditioning zmatrix size %d zblock size %d zNo constraints z%d constraints z%d constraint zThe problem size z minus the constraints size z) is too small relative to the block size zd. Using a dense eigensolver instead of LOBPCG iterations.No output of the history of the iterations.z3The dense eigensolver does not support constraints.r=z> of the primary matrix defined by a callable object is wrong. z&Primary MatMul call failed with error r;z@ of the secondary matrix defined by a callable object is wrong. r:Zsubset_by_indexeigvals check_finiteFrEz$Dense eigensolver failed with error gz0 of the constraint not preserved and changed to r>zLinearly dependent constraints)rBz)Linearly dependent initial approximationsz< of the initial approximations not preserved and changed to z* after multiplying by the primary matrix. )rSTz7 of the restarted iterate not preserved and changed to z iteration zcurrent block size: zeigenvalue(s): zresidual norm(s): zFailed at iteration z with accuracies z' not reaching the requested tolerance .float32zeigh failed at iteration z with error z causing a restart. z with error zExited at iteration z with accuracies z& not reaching the requested tolerance z. Use iteration z instead with accuracy z. zFinal iterative eigenvalue(s): z"Final iterative residual norm(s): z< of the postprocessing iterate not preserved and changed to z(eigh has failed in lobpcg postprocessingz'Exited postprocessing with accuracies z$Final postprocessing eigenvalue(s): zFinal residual norm(s): cSsg|]}t|qSr rZsqueeze.0ir r r! szlobpcg..cSsg|]}t|qSr rVrWr r r!rZs)5lenr'rrrr@rZ issubdtyperZinexactZasarrayrUzerosprintminNotImplementedError isinstancerZeyeintr2r Ztoarrayr?inspect signaturer parameterssqrtrrr3r4rrrrr9rDrJZonesboolabsrZnewaxissumwherer*ZdiagZastypebmatrMZvsplit)OAXrArYrmaxiterrHrBZretLambdaHistoryZretResidualNormsHistoryZrestartControlZ blockVectorXZbestblockVectorXr7ZresidualToleranceZbestIterationNumberZsizeYnZsizeXZ lambdaHistoryZresidualNormsHistoryr)rRrCZadditional_paramsvalsZvecsr6ZgramYBYZ blockVectorBX_Z blockVectorAXZgramXAXrFZeigBlockVectorrIZ activeMaskZ blockVectorPZ blockVectorAPZ blockVectorBPZsmallestResidualNormZiterationNumberZrestartZ forcedRestartZexplicitGramFlagZ blockVectorRZ residualNormsZ residualNormZcurrentBlockSizeZactiveBlockVectorRZactiveBlockVectorPZactiveBlockVectorAPZactiveBlockVectorBPZactiveBlockVectorBRZactiveBlockVectorARZinvRnormalZmyepsZgramXARZgramRARZ gramDtypeZgramXBXZgramRBRZgramXBRZgramXAPZgramRAPZgramPAPZgramXBPZgramRBPZgramPBPrKrLZeigBlockVectorXZeigBlockVectorRZeigBlockVectorPppZappZbppr r r!r sD                                                                                     "             )Nr) NNNNNTrFFrN)__doc__rbrZnumpyrrrrrrrrZscipy.sparse.linalgrZ scipy.sparser r rj__all__r"r*r3r9rDrJrMr r r r r!s8       0