U -e@sdZddlZddlmZddlZddlmZmZm Z ddl m Z ddl m Z mZdd lmZmZdd lmZmZmZmZmZmZd d Zd dZGdddeZdS)z Bayesian Gaussian Mixture Model.N)Real)betalndigammagammaln) check_array)Interval StrOptions) BaseMixture _check_shape)_check_precision_matrix_check_precision_positivity_compute_log_det_cholesky_compute_precision_cholesky_estimate_gaussian_parameters_estimate_log_gaussian_probcCstt|tt|S)aVCompute the log of the Dirichlet distribution normalization term. Parameters ---------- dirichlet_concentration : array-like of shape (n_samples,) The parameters values of the Dirichlet distribution. Returns ------- log_dirichlet_norm : float The log normalization of the Dirichlet distribution. )rnpsum)Zdirichlet_concentrationrb/var/www/html/Darija-Ai-Train/env/lib/python3.8/site-packages/sklearn/mixture/_bayesian_mixture.py_log_dirichlet_norms rc CsL||||dtdttd|t|ddtjfd S)aDCompute the log of the Wishart distribution normalization term. Parameters ---------- degrees_of_freedom : array-like of shape (n_components,) The number of degrees of freedom on the covariance Wishart distributions. log_det_precision_chol : array-like of shape (n_components,) The determinant of the precision matrix for each component. n_features : int The number of features. Return ------ log_wishart_norm : array-like of shape (n_components,) The log normalization of the Wishart distribution. ?@Nr)mathlogrrrarangenewaxis)Zdegrees_of_freedomlog_det_precisions_chol n_featuresrrr_log_wishart_norm+s"r cs`eZdZUdZejeddddhgeddhgdeed dd d gdeed dd d gdd gdeed dd d gdd eed dd d gd Ze e d<ddddddddddddddddddfdd Z ddZ ddZ ddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zd.d/Zd0d1Zd2d3Zd4d5Zd6d7Zd8d9Zd:d;Zdd?ZZS)@BayesianGaussianMixturea)/Variational Bayesian estimation of a Gaussian mixture. This class allows to infer an approximate posterior distribution over the parameters of a Gaussian mixture distribution. The effective number of components can be inferred from the data. This class implements two types of prior for the weights distribution: a finite mixture model with Dirichlet distribution and an infinite mixture model with the Dirichlet Process. In practice Dirichlet Process inference algorithm is approximated and uses a truncated distribution with a fixed maximum number of components (called the Stick-breaking representation). The number of components actually used almost always depends on the data. .. versionadded:: 0.18 Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=1 The number of mixture components. Depending on the data and the value of the `weight_concentration_prior` the model can decide to not use all the components by setting some component `weights_` to values very close to zero. The number of effective components is therefore smaller than n_components. covariance_type : {'full', 'tied', 'diag', 'spherical'}, default='full' String describing the type of covariance parameters to use. Must be one of:: 'full' (each component has its own general covariance matrix), 'tied' (all components share the same general covariance matrix), 'diag' (each component has its own diagonal covariance matrix), 'spherical' (each component has its own single variance). tol : float, default=1e-3 The convergence threshold. EM iterations will stop when the lower bound average gain on the likelihood (of the training data with respect to the model) is below this threshold. reg_covar : float, default=1e-6 Non-negative regularization added to the diagonal of covariance. Allows to assure that the covariance matrices are all positive. max_iter : int, default=100 The number of EM iterations to perform. n_init : int, default=1 The number of initializations to perform. The result with the highest lower bound value on the likelihood is kept. init_params : {'kmeans', 'k-means++', 'random', 'random_from_data'}, default='kmeans' The method used to initialize the weights, the means and the covariances. String must be one of: 'kmeans' : responsibilities are initialized using kmeans. 'k-means++' : use the k-means++ method to initialize. 'random' : responsibilities are initialized randomly. 'random_from_data' : initial means are randomly selected data points. .. versionchanged:: v1.1 `init_params` now accepts 'random_from_data' and 'k-means++' as initialization methods. weight_concentration_prior_type : {'dirichlet_process', 'dirichlet_distribution'}, default='dirichlet_process' String describing the type of the weight concentration prior. weight_concentration_prior : float or None, default=None The dirichlet concentration of each component on the weight distribution (Dirichlet). This is commonly called gamma in the literature. The higher concentration puts more mass in the center and will lead to more components being active, while a lower concentration parameter will lead to more mass at the edge of the mixture weights simplex. The value of the parameter must be greater than 0. If it is None, it's set to ``1. / n_components``. mean_precision_prior : float or None, default=None The precision prior on the mean distribution (Gaussian). Controls the extent of where means can be placed. Larger values concentrate the cluster means around `mean_prior`. The value of the parameter must be greater than 0. If it is None, it is set to 1. mean_prior : array-like, shape (n_features,), default=None The prior on the mean distribution (Gaussian). If it is None, it is set to the mean of X. degrees_of_freedom_prior : float or None, default=None The prior of the number of degrees of freedom on the covariance distributions (Wishart). If it is None, it's set to `n_features`. covariance_prior : float or array-like, default=None The prior on the covariance distribution (Wishart). If it is None, the emiprical covariance prior is initialized using the covariance of X. The shape depends on `covariance_type`:: (n_features, n_features) if 'full', (n_features, n_features) if 'tied', (n_features) if 'diag', float if 'spherical' random_state : int, RandomState instance or None, default=None Controls the random seed given to the method chosen to initialize the parameters (see `init_params`). In addition, it controls the generation of random samples from the fitted distribution (see the method `sample`). Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. warm_start : bool, default=False If 'warm_start' is True, the solution of the last fitting is used as initialization for the next call of fit(). This can speed up convergence when fit is called several times on similar problems. See :term:`the Glossary `. verbose : int, default=0 Enable verbose output. If 1 then it prints the current initialization and each iteration step. If greater than 1 then it prints also the log probability and the time needed for each step. verbose_interval : int, default=10 Number of iteration done before the next print. Attributes ---------- weights_ : array-like of shape (n_components,) The weights of each mixture components. means_ : array-like of shape (n_components, n_features) The mean of each mixture component. covariances_ : array-like The covariance of each mixture component. The shape depends on `covariance_type`:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' precisions_ : array-like The precision matrices for each component in the mixture. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on ``covariance_type``:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' precisions_cholesky_ : array-like The cholesky decomposition of the precision matrices of each mixture component. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on ``covariance_type``:: (n_components,) if 'spherical', (n_features, n_features) if 'tied', (n_components, n_features) if 'diag', (n_components, n_features, n_features) if 'full' converged_ : bool True when convergence was reached in fit(), False otherwise. n_iter_ : int Number of step used by the best fit of inference to reach the convergence. lower_bound_ : float Lower bound value on the model evidence (of the training data) of the best fit of inference. weight_concentration_prior_ : tuple or float The dirichlet concentration of each component on the weight distribution (Dirichlet). The type depends on ``weight_concentration_prior_type``:: (float, float) if 'dirichlet_process' (Beta parameters), float if 'dirichlet_distribution' (Dirichlet parameters). The higher concentration puts more mass in the center and will lead to more components being active, while a lower concentration parameter will lead to more mass at the edge of the simplex. weight_concentration_ : array-like of shape (n_components,) The dirichlet concentration of each component on the weight distribution (Dirichlet). mean_precision_prior_ : float The precision prior on the mean distribution (Gaussian). Controls the extent of where means can be placed. Larger values concentrate the cluster means around `mean_prior`. If mean_precision_prior is set to None, `mean_precision_prior_` is set to 1. mean_precision_ : array-like of shape (n_components,) The precision of each components on the mean distribution (Gaussian). mean_prior_ : array-like of shape (n_features,) The prior on the mean distribution (Gaussian). degrees_of_freedom_prior_ : float The prior of the number of degrees of freedom on the covariance distributions (Wishart). degrees_of_freedom_ : array-like of shape (n_components,) The number of degrees of freedom of each components in the model. covariance_prior_ : float or array-like The prior on the covariance distribution (Wishart). The shape depends on `covariance_type`:: (n_features, n_features) if 'full', (n_features, n_features) if 'tied', (n_features) if 'diag', float if 'spherical' n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- GaussianMixture : Finite Gaussian mixture fit with EM. References ---------- .. [1] `Bishop, Christopher M. (2006). "Pattern recognition and machine learning". Vol. 4 No. 4. New York: Springer. `_ .. [2] `Hagai Attias. (2000). "A Variational Bayesian Framework for Graphical Models". In Advances in Neural Information Processing Systems 12. `_ .. [3] `Blei, David M. and Michael I. Jordan. (2006). "Variational inference for Dirichlet process mixtures". Bayesian analysis 1.1 `_ Examples -------- >>> import numpy as np >>> from sklearn.mixture import BayesianGaussianMixture >>> X = np.array([[1, 2], [1, 4], [1, 0], [4, 2], [12, 4], [10, 7]]) >>> bgm = BayesianGaussianMixture(n_components=2, random_state=42).fit(X) >>> bgm.means_ array([[2.49... , 2.29...], [8.45..., 4.52... ]]) >>> bgm.predict([[0, 0], [9, 3]]) array([0, 1]) sphericaltieddiagfulldirichlet_processZdirichlet_distributionNgZneither)closedz array-like)covariance_typeweight_concentration_prior_typeweight_concentration_priormean_precision_prior mean_priordegrees_of_freedom_priorcovariance_prior_parameter_constraintsr gMbP?gư>dZkmeansFr ) n_componentsr(tol reg_covarmax_itern_init init_paramsr)r*r+r,r-r. random_state warm_startverboseverbose_intervalc sNtj||||||||||d ||_||_| |_| |_| |_| |_| |_dS)N) r2r3r4r5r6r7r8r9r:r;) super__init__r(r)r*r+r,r-r.)selfr2r(r3r4r5r6r7r)r*r+r,r-r.r8r9r:r; __class__rrr=ps& z BayesianGaussianMixture.__init__cCs*|||||||dS)zCheck that the parameters are well defined. Parameters ---------- X : array-like of shape (n_samples, n_features) N)_check_weights_parameters_check_means_parameters_check_precision_parameters _checkcovariance_prior_parameter)r>Xrrr_check_parameterss  z)BayesianGaussianMixture._check_parameterscCs$|jdkrd|j|_n|j|_dS)z2Check the parameter of the Dirichlet distribution.N?)r*r2weight_concentration_prior_r>rrrrAs z1BayesianGaussianMixture._check_weights_parameterscCsl|j\}}|jdkrd|_n|j|_|jdkr>|jdd|_n*t|jtjtj gdd|_t |j|fddS)zCheck the parameters of the Gaussian distribution. Parameters ---------- X : array-like of shape (n_samples, n_features) NrGr)axisFZdtypeZ ensure_2dZmeans) shaper+mean_precision_prior_r,mean mean_prior_rrfloat64float32r r>rE_rrrrrBs    z/BayesianGaussianMixture._check_means_parameterscCsN|j\}}|jdkr||_n.|j|dkr4|j|_ntd|d|jfdS)zCheck the prior parameters of the precision distribution. Parameters ---------- X : array-like of shape (n_samples, n_features) NrGzQThe parameter 'degrees_of_freedom_prior' should be greater than %d, but got %.3f.r )rLr-degrees_of_freedom_prior_ ValueErrorrRrrrrCs    z3BayesianGaussianMixture._check_precision_parameterscCs|j\}}|jdkrdtt|jtt|jtj|dddtj|dddd|j|_ n|jdkrt |jtj tj gdd|_ t |j ||fd |jt|j |jnR|jd krt |jtj tj gdd|_ t |j |fd |jt|j |jn|j|_ dS) zCheck the `covariance_prior_`. Parameters ---------- X : array-like of shape (n_samples, n_features) Nrr )rJZddofr%r#r$r")r%r#FrKz%s covariance_priorr$)rLr.rZ atleast_2dZcovTvarrNr(covariance_prior_rrPrQr r rrRrrrrDsD      z8BayesianGaussianMixture._checkcovariance_prior_parametercCs@t|||j|j\}}}|||||||||dS)zInitialization of the mixture parameters. Parameters ---------- X : array-like of shape (n_samples, n_features) resp : array-like of shape (n_samples, n_components) N)rr4r(_estimate_weights_estimate_means_estimate_precisions)r>rErespnkxkskrrr _initializes    z#BayesianGaussianMixture._initializec CsT|jdkrDd||jtt|dddddddff|_n |j||_dS)zEstimate the parameters of the Dirichlet distribution. Parameters ---------- nk : array-like of shape (n_components,) r&rGNr)r)rHrhstackcumsumweight_concentration_)r>r^rrrrZs & z)BayesianGaussianMixture._estimate_weightscCsH|j||_|j|j|ddtjf||jddtjf|_dS)zEstimate the parameters of the Gaussian distribution. Parameters ---------- nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) N)rMmean_precision_rOrrmeans_)r>r^r_rrrr[&s  z'BayesianGaussianMixture._estimate_meanscCs8|j|j|j|jd|j|||t|j|j|_dS)aEstimate the precisions parameters of the precision distribution. Parameters ---------- nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) sk : array-like The shape depends of `covariance_type`: 'full' : (n_components, n_features, n_features) 'tied' : (n_features, n_features) 'diag' : (n_components, n_features) 'spherical' : (n_components,) rVN)_estimate_wishart_full_estimate_wishart_tied_estimate_wishart_diag_estimate_wishart_sphericalr(r covariances_precisions_cholesky_)r>r^r_r`rrrr\4sz,BayesianGaussianMixture._estimate_precisionscCs|j\}}|j||_t|j||f|_t|jD]R}|||j}|j |||||||j |j |t |||j|<q4|j|jddtj tj f_dS)aREstimate the full Wishart distribution parameters. Parameters ---------- X : array-like of shape (n_samples, n_features) nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) sk : array-like of shape (n_components, n_features, n_features) N)rLrTdegrees_of_freedom_remptyr2rmrangerOrYrMrgouterr)r>r^r_r`rSrkdiffrrrriOs$    z.BayesianGaussianMixture._estimate_wishart_fullcCs||j\}}|j||j|_||j}|j|||j|j|jt ||j |j ||_ |j |j_ dS)aDEstimate the tied Wishart distribution parameters. Parameters ---------- X : array-like of shape (n_samples, n_features) nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) sk : array-like of shape (n_features, n_features) N) rLrTrr2rorOrYrMrdotrgrWrmr>r^r_r`rSrrtrrrrjss   z.BayesianGaussianMixture._estimate_wishart_tiedcCs|j\}}|j||_||j}|j|ddtjf||j|jddtjft ||_ |j |jddtjf_ dS)aFEstimate the diag Wishart distribution parameters. Parameters ---------- X : array-like of shape (n_samples, n_features) nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) sk : array-like of shape (n_components, n_features) N) rLrTrorOrYrrrMrgsquarermrvrrrrks   z.BayesianGaussianMixture._estimate_wishart_diagc Csb|j\}}|j||_||j}|j|||j|jtt |d|_ |j |j_ dS)a@Estimate the spherical Wishart distribution parameters. Parameters ---------- X : array-like of shape (n_samples, n_features) nk : array-like of shape (n_components,) xk : array-like of shape (n_components, n_features) sk : array-like of shape (n_components,) r N) rLrTrorOrYrMrgrrNrwrmrvrrrrls   z3BayesianGaussianMixture._estimate_wishart_sphericalcCsP|j\}}t|t||j|j\}}}|||||||||dS)a*M step. Parameters ---------- X : array-like of shape (n_samples, n_features) log_resp : array-like of shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. N) rLrrexpr4r(rZr[r\)r>rElog_respZ n_samplesrSr^r_r`rrr_m_steps    zBayesianGaussianMixture._m_stepcCs|jdkrft|jd|jd}t|jd}t|jd}||tdt||ddfSt|jtt|jSdS)Nr&rr rb)r)rrfrrdrer)r>Z digamma_sumZ digamma_aZ digamma_brrr_estimate_log_weightss   z-BayesianGaussianMixture._estimate_log_weightsc Cs|j\}}t||j|j|jd|t|j}|tdtt d|jt d|ddtj fd}|d|||j S)Nrrr) rLrrhrnr(rrrorrrrrg)r>rErSrZ log_gaussZ log_lambdarrr_estimate_log_probs$ z*BayesianGaussianMixture._estimate_log_probcCs|jj\}t|j|j|d|t|j}|jdkrT|jt t |j||}nt t |j||}|j dkrt t |jd|jd }n t|j}t t|| ||d|t t|jS)aEstimate the lower bound of the model. The lower bound on the likelihood (of the training data with respect to the model) is used to detect the convergence and has to increase at each iteration. Parameters ---------- X : array-like of shape (n_samples, n_features) log_resp : array, shape (n_samples, n_components) Logarithm of the posterior probabilities (or responsibilities) of the point of each sample in X. log_prob_norm : float Logarithm of the probability of each sample in X. Returns ------- lower_bound : float rr#r&rr )rOrLrrnr(rrror2rPr rr)rrfrrxrg)r>ryZ log_prob_normrrZ log_wishartZlog_norm_weightrrr_compute_lower_bound sH    z,BayesianGaussianMixture._compute_lower_boundcCs|j|j|j|j|j|jfS)N)rfrgrhrormrnrIrrr_get_parametersEsz'BayesianGaussianMixture._get_parametersc Cs|\|_|_|_|_|_|_|jdkr|jd|jd}|jd|}|jd|tdt |ddf|_ |j t |j _ n|jt |j|_ |j dkrt dd|jD|_n,|j dkrt|j|jj|_n |jd |_dS) Nr&rr rbr%cSsg|]}t||jqSr)rrurW).0Z prec_cholrrr msz;BayesianGaussianMixture._set_parameters..r#r)rfrgrhrormrnr)rrdZcumprodZweights_rr(arrayZ precisions_rurW)r>paramsZweight_dirichlet_sumtmprrr_set_parametersOsD   z'BayesianGaussianMixture._set_parameters) __name__ __module__ __qualname____doc__r r/r rrdict__annotations__r=rFrArBrCrDrarZr[r\rirjrkrlrzr{r|r}r~r __classcell__rrr?rr!Jsn  * )$"; r!)rrnumbersrnumpyrZ scipy.specialrrrutilsrZutils._param_validationrr _baser r Z_gaussian_mixturer rrrrrrr r!rrrrs