U 2de@s6dZddlZddlmZddlmZGdddZdS)zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) squared_normc@sleZdZdZddZdddZddZd d Zd d ZdddZ dddZ dddZ dddZ dddZ dS)LinearModelLossaGeneral class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the sum of per sample losses and includes a term for L2 regularization:: loss = sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None. Gradient and hessian, for simplicity without intercept, are:: gradient = X.T @ loss.gradient + l2_reg_strength * coef hessian = X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_features::n_dof] = coef[(n_dof-1)::n_dof] intercept.shape = (n_classes,) else: intercept = coef[-1] Note: If coef has shape (n_classes * n_dof,), the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool cCs||_||_dS)N) base_loss fit_intercept)selfrrr E/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/linear_model/_linear_loss.py__init__BszLinearModelLoss.__init__NcCsZ|jd}|jj}|jr"|d}n|}|jjrFtj|||f|dd}ntj|||d}|S)aAllocate coef of correct shape with zeros. Parameters: ----------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. dtype : data-type, default=None Overrides the data type of coef. With dtype=None, coef will have the same dtype as X. Returns ------- coef : ndarray of shape (n_dof,) or (n_classes, n_dof) Coefficients of a linear model. F)shapedtypeorderrr)rr n_classesr is_multiclassnpZ zeros_like)rXr n_featuresrn_dofcoefr r r init_zero_coefFs  zLinearModelLoss.init_zero_coefcCs|jjs.|jr$|d}|dd}qd}|}nV|jdkrP|j|jjdfdd}n|}|jr|dddf}|ddddf}nd}||fS)aHelper function to get coefficients and intercept. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. Nr r r)rrrndimreshaper)rr interceptweightsr r r weight_interceptbs z LinearModelLoss.weight_interceptcCs<||\}}|jjs$|||}n||j|}|||fS)aiHelper function to get coefficients, intercept and raw_prediction. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. Returns ------- weights : ndarray of shape (n_features,) or (n_classes, n_features) Coefficients without intercept term. intercept : float or ndarray of shape (n_classes,) Intercept terms. raw_prediction : ndarray of shape (n_samples,) or (n_samples, n_classes) )r!rrT)rrrr rraw_predictionr r r weight_intercept_raws z$LinearModelLoss.weight_intercept_rawcCs&|jdkr||nt|}d||S)z5Compute L2 penalty term l2_reg_strength/2 *||w||_2^2.r g?)rr)rr l2_reg_strengthZnorm2_wr r r l2_penaltyszLinearModelLoss.l2_penaltyrr c CsV|dkr|||\}} }n||\}} |jj||||d} | } | |||S)aCompute the loss as sum over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Sum of losses per sample plus penalty. NZy_truer# sample_weight n_threads)r$r!rlosssumr&) rrryr(r%r)r#r rr*r r r r*s'zLinearModelLoss.losscCs:|jd|jj}} |t|j} |dkr>|||\} } }n||\} } |jj||||d\} }| } | | | |7} |jj st j || j d}|j||| |d|<|jr||d<npt j| | f| j dd}|j||| |ddd|f<|jr|jdd |dddf<|jdkr2|jdd }| |fS) aNComputes the sum of loss and gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Sum of losses per sample plus penalty. gradient : ndarray of shape coef.shape The gradient of the loss. r Nr'rrr rrrZaxisr)rrrintrr$r! loss_gradientr+r&rr empty_likerr"emptyrravel)rrrr,r(r%r)r#rrrr rr*grad_pointwisegradr r r r1s2* "  zLinearModelLoss.loss_gradientcCs |jd|jj}} |t|j} |dkr>|||\} } }n||\} } |jj||||d} |jjst j || j d}|j | || |d|<|jr| |d<|St j| | f| j dd}| j ||| |ddd|f<|jr| j dd |dddf<|jdkr|jdd S|SdS) aComputes the gradient w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. r Nr'r-rr r.rr/r)rrrr0rr$r!gradientrrr2rr"r+r3rr4)rrrr,r(r%r)r#rrrr rr5r6r r r r7/s0' "  zLinearModelLoss.gradientc Cs|j\} } | t|j} | dkr4|||\} }} n||\} }|jj|| ||d\}}t|dkdk}t |}|jj s|dkrtj || j d}n|}|j ||| |d| <|jr||d<|dkrtj| | f| j d}n|}|r|||fSt|r<|j tj|df| | fd||d| d| f<n2|dddf|}t|j ||d| d| f<|dkr|dd| | | d |7<|jr|j |}||dddf<||dddf<||d <nt|||fS) aComputes gradient and hessian w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. gradient_out : None or ndarray of shape coef.shape A location into which the gradient is stored. If None, a new array might be created. hessian_out : None or ndarray A location into which the hessian is stored. If None, a new array might be created. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessian : ndarray Hessian matrix. hessian_warning : bool True if pointwise hessian has more than half of its elements non-positive. Nr'rg?r-rrrr )rr)rr0rr$r!rgradient_hessianrZmeanabsrr2rr"r+r3rissparse dia_matrixZtoarraydotrNotImplementedError)rrrr,r(r%r)Z gradient_outZ hessian_outr# n_samplesrrr rr5hess_pointwiseZhessian_warningr6ZhessZWXZXhr r r r9vsf5           z LinearModelLoss.gradient_hessianc sj jj\}t j \ }} jjs jj|| |d\} } tj  j d} j |  | d< jr| | d<| t rt j| df||fdn| ddtjf jrttj ddt fdd } n jj|| |d\} tjf j d d } | j  | dddf< jr| j dd| dddf< f d d } jd kr| jd d| fS| | fS)aComputes gradient and hessp (hessian product function) w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessp : callable Function that takes in a vector input of shape of gradient and and returns matrix-vector product with hessian. r'r-Nrrr8r/cst|}tr4j|d|d<n$tjj|dg|d<|d|d7<jr|d|d7<|d|d|d<|S)Nr)rr2rr;r"ZlinalgZ multi_dotr)sret)rhXhX_sum hessian_sumr%rrr r hesspDs   $  z7LinearModelLoss.gradient_hessian_product..hesspr r.cs|jdfdd}jr>|dddf}|ddddf}nd}|j|}| |jddddtjf7}|9}dk r|ddtjf9}tjf jdd}|j||dddf<jr|jdd|dddf<jdkr|j ddS|SdS)Nrr rrr r/r.) rrr"r+rnewaxisr3rrr4)rAZ s_intercepttmpZ hess_prod) rrr%rrrprobar(rr r r rFws"$"  r r)rrrr0rr$rr9rr2rr"r+rr;r<rGZsqueezeZasarrayZ atleast_1dZgradient_probar3rr4)rrrr,r(r%r)r?rr#r5r@r6rFr ) rrrCrDrEr%rrrrIr(rr r gradient_hessian_productsN       " z(LinearModelLoss.gradient_hessian_product)N)Nrr N)Nrr N)Nrr N)Nrr NNN)Nrr )__name__ __module__ __qualname____doc__r rr!r$r&r*r1r7r9rJr r r r r sB8 '   ; P L  r)rNZnumpyrZscipyrZ utils.extmathrrr r r r s