U 3ds@s(ddlmZddlmZddlmZddlmZddlZddlZddlZ ddl m Z ddl m Z ddlZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddl m!Z!ddl"m#Z#ddl"m$Z$ddl"m%Z%ddl"m&Z&ddl"m'Z'ddl"m(Z(ddl"m)Z)ddl"m*Z*ddl"m+Z+ddl"m,Z,ddl"m-Z-ddl"m.Z.dd l"m/Z/dd!l"m0Z0dd"l"m1Z1dd#l"m2Z2dd$l"m3Z3dd%l"m4Z4dd&l"m5Z5dd'l"m6Z6dd(l7m8Z8dd)l9m:Z:dd*l;md.d/Z?d0d1Z@ejABd2d3dd4gd5d6ZCd7d8ZDd9d:ZEed;d<ZFed=d>ZGed?d@ZHdAdBZIdCdDZJdEdFZKedGdHZLdIdJZMdKdLZNdMdNZOdOdPZPdQdRZQdSdTZRejABdUdVdWdXdYgdZd[ZSd\d]ZTd^d_ZUejABd`e Vddddddge Vddddddgdadbfe Vd4d4d4dddge Vd4d4d4dddgdadcfe Vd4d4d4dddge Vddddddgdaddfe Vd4d4d4dddge Vdddd4d4d4gdadefe Vddddddge Vd4d4d4dddgdadffgdgdhZWejABdie Vdd4dd4dge Vd4d4dddjgdadkfgdldmZXdndoZYdpdqZZdrdsZ[dtduZ\dvdwZ]dxdyZ^dzd{Z_ejABd|d}d~gddZ`ddZaejABddddddgddZbddZcddZdddZeejAjBdgdfddgdfgddgdddZfejAjBdddd4gdd4djgfdddgdddZgddZhddZiddZjddZkddZlddZmddZnddZoddZpddZqeddZrddZsddZtddZuddZvddZwddZxddÄZyejABdddgddȄZzeddʄZ{edd̄Z|eejABd2d3dd4gdd΄Z}ejABdd4gejABdddddgejABd2dd4gddфZ~ejABdddddgddӄZejABd2dd4gddՄZddׄZddلZejABd2dd4gddۄZejABd2d3dd4gdd݄ZejABd2d3dd4gdd߄ZejABd2d3dd4gddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZejABdddddgddddgfddddgddddgfddddgddddgfgddZdS( )partial)product)chain) permutationsN)linalg) bernoulli)datasets)svm)make_multilabel_classification)label_binarizeLabelBinarizer)check_random_state)assert_almost_equal)assert_array_equal)assert_array_almost_equal)assert_allclose)assert_no_warnings)ignore_warnings) MockDataFrame)accuracy_score)average_precision_score)balanced_accuracy_score)class_likelihood_ratios)classification_report)cohen_kappa_score)confusion_matrix)f1_score) fbeta_score) hamming_loss) hinge_loss) jaccard_score)log_loss)matthews_corrcoef)precision_recall_fscore_support)precision_score) recall_score) zero_one_loss)brier_score_loss)multilabel_confusion_matrix)_check_targets)UndefinedMetricWarning)hammingFc Cs|dkrt}|j}|j}|r:||dk||dk}}|j\}}t|}td}||||||}}t |d}tj d}tj || |d|f}tjdddd} | |d||d|||d} |r| ddd f} | ||d} ||d} | | | fS) zMake some classification predictions on a toy dataset using a SVC If binary is True restrict to a binary classification problem instead of a multiclass classification problem N%rlinearT)kernelZ probability random_state)r load_irisdatatargetshapenparanger shuffleintrandom RandomStateZc_Zrandnr ZSVCfitZ predict_probaZpredict) datasetbinaryXy n_samples n_featuresprngZhalfZclfZ probas_predy_predy_truerHM/tmp/pip-unpacked-wheel-zrfo1fqw/sklearn/metrics/tests/test_classification.pymake_prediction9s*     * rJc Cs|t}t|dd\}}}ddddddd d d dd d ddddddddddddddd}t||tt|j|jdd}||kst |D]x}|dkrt ||t st ||||kst q||||kst ||D]}t ||||||qqt |ddt ks*t t |ddt ksDt t |ddtks^t t |ddtksxt dS) NFr>r?g7Mo?gUUUUUU?ghQ?) precisionrecallf1-scoresupportUUUUUU?gc1Ƹ?g433333?g)k??$I$I?gCFQ?gc?gƢ?KrOrMrNrPg?gD~WG?g]3p?)setosaZ versicolorZ virginica macro avgaccuracy weighted avgT)labels target_names output_dictrZrXrMrYrP)rr3rJrr7r8lenr]keysAssertionError isinstancefloatrtyper:)irisrGrF_expected_reportreportkeymetricrHrHrI,test_classification_report_dictionary_outputis`"  rkcCstggdd}dtjtjtjdddddddd}t|tsBt||ksVt|D]x}|dkrt||tsxt||||kstqZ||||kst||D]}t||||||qqZdS)NT)rGrFr^rrW)rZrYr[rZ) rr7nanrbdictrar`rcr)rhrgrirjrHrHrI2test_classification_report_output_dict_empty_inputs, ro zero_divisionwarnr2c Csdddgdddg}}tjddX}t|||dd|dkrnt|d ksLt|D]}d }|t|jksPtqPn|rvtW5QRXdS) NabcdTrecord)rpr^rqr2z7Use `zero_division` parameter to control this behavior.)warningscatch_warningsrr_rastrmessage)rprGrFrwitemmsgrHrHrI0test_classification_report_zero_division_warningsr~cCstdddgdddgg}tdddgdddgg}t||dksFtt||dksXtt||dksjtt|t|dkstt|t|dkstt|t|jdkstt|t|jdkstdSNrr2?)r7arrayrra logical_notzerosr6y1y2rHrHrI.test_multilabel_accuracy_score_subset_accuracysrc Cstdd\}}}t||dd\}}}}t|ddgdt|ddgdt|d d gdt|d d gitfd d itffD]\}}|t||f|} t| dd|t||f|} t| dd|t||f|} t| d dt|t ||fddi|d| | d| | dqxdS)NTr?averageg\(\?g333333?r,g)\(?g(\?皙?gRQ?rr?beta) rJr#rrrr$r%rrr) rGrFrfrDrfskwargsZ my_assertpsrsfsrHrHrI%test_precision_recall_f1_score_binarys(    rcCsdtddgddgkstdtddgddgks4tdtddgddgksNtdtddgddgddksltdtddgddgkstdtddgddgkstdtddgddgkstdtddgddgtddksttddgddgtddttddgddgddkstdS) N?r2rrrlinfgj@)r$rar%rrrcpytestapproxrHrHrHrI+test_precision_recall_f_binary_single_class s"rc Csddddg}ddddg}t|tdd}t|tdd}||f||fg}t|D]\}\}}t||dddddgdd}td d d d d g|t||dddddgd d}ttd d d d d g|d D]B}|dkr|dkrqtt||dddddg|dt||d|dqqTdD]`}t t t||td|dW5QRXt t t||tdd|dW5QRXqt dddgdddgg}t dddgdddgg}t ||dddgd\}} } } tt || | gt dddgdS)Nr2r,rclassesrrr\rrlrrmacro)microweightedsamplesr)Nrrrrrr\?竪?) r r7r8 enumerater%rmeanrrraises ValueErrorrr#) rGrF y_true_bin y_pred_binr4iactualrrDrrrfrHrHrI$test_precision_recall_f_extra_labelssH      rc Csddddg}ddddg}t|tdd}t|tdd}||f||fg}t|D]\}\}}tt||ddgd}tt||dd}tddg|dd td |d d td |d d td |dd dD]}||d ||d kstqqTdS)Nr2r,rrrr\rrrrrUUUUUU?rr)rrr) r r7r8rrr%rrra) rGrFrrr4rZ recall_13Z recall_allrrHrHrI&test_precision_recall_f_ignored_labelsJs  rc CsNtd}|d}|jdddd}d}tjt|dt||W5QRXdS)Ni rrsizez"multiclass format is not supportedmatch)r randrandintrrrr)rErFrGerr_msgrHrHrI3test_average_precision_score_score_non_binary_classas  rc CsJdddddddddddg }dddddddddddg }t||dksFtdS)Nrr2皙?皙?r333333?rSrrarGy_scorerHrHrI-test_average_precision_score_duplicate_valuesnsrcCs*dddg}dddg}t||dks&tdS)Nrr2rrrrrrHrHrI(test_average_precision_score_tied_valuesys  rc Cstdd\}}}ttt||ddW5QRXttt||dddW5QRXtt tdd dgd ddgd d W5QRXdS) NTrgrr,r? pos_labelrrr2Zmegar)rJrrrr#)rGrFrfrHrHrI+test_precision_recall_fscore_support_errorss   rc Cs>d}tjt|d"tdddgdddgdddW5QRXdS)NzNote that pos_label \(set to 2\) is ignored when average != 'binary' \(got 'macro'\). You may use labels=\[pos_label\] to specify a single positive class.rr2r,rr)rwarns UserWarningr#r}rHrHrI(test_precision_recall_f_unused_pos_labelsrcCsDtdd\}}}dd}||||dd|Ddd|DdS)NTrc Sst||}t|ddgddgg|\}}}}||||}t||||||||}|dkrrdn||} t||} t| | ddt| ddddS) Nrrr,decimal= ףp=?)rrflattenr7sqrtr"r) rGrFcmtpfpfntnnumZdenZtrue_mccZmccrHrHrItests & z*test_confusion_matrix_binary..testcSsg|] }t|qSrHrz.0rArHrHrI sz0test_confusion_matrix_binary..rJrGrFrfrrHrHrItest_confusion_matrix_binarys rcCsDtdd\}}}dd}||||dd|Ddd|DdS)NTrcSs4t||}t|ddgddggddgddgggdS)Nrrrrr(r)rGrFrrHrHrIrs z5test_multilabel_confusion_matrix_binary..testcSsg|] }t|qSrHrrrHrHrIrsz;test_multilabel_confusion_matrix_binary..rrrHrHrI'test_multilabel_confusion_matrix_binarys rcCsJtdd\}}}d dd}||||dd|Ddd|DdddS) NFrcSst||}t|ddgddggddgddggd d gd d ggg|rLd ddgndd dg}t|||d}t|ddgddggd d gd d ggddgddggg|rd dddgn dd ddg}t|||d}t|ddgddggd d gd d ggddgddggddgddgggdS)N/rr&rrrr,021rr2r3rVr)rGrF string_typerr\rHrHrIrs* ,,z9test_multilabel_confusion_matrix_multiclass..testcSsg|] }t|qSrHrrrHrHrIrsz?test_multilabel_confusion_matrix_multiclass..T)r)FrrrHrHrI+test_multilabel_confusion_matrix_multiclasss  rcCsddlm}m}tdddgdddgdddgg}tdddgdddgdddgg}||}||}||}||}tdddg}ddgddggddgddggddgddggg} |||g} |||g} | D]"} | D]} t| | }t|| qqt||dd}t|ddgddggddgddggddgddgggt||ddgd}t|ddgddggddgddgggt||ddgdd }t|ddgddggddgddggddgddgggt|||dd }t|ddgddggddgddggddgd dgggdS) Nr) csc_matrix csr_matrixr2r,rT samplewiser)r\r) sample_weightrr)Z scipy.sparserrr7rr(r)rrrGrFZ y_true_csrZ y_pred_csrZ y_true_cscZ y_pred_cscrZreal_cmZtruespredsZ y_true_tmpZ y_pred_tmprrHrHrI+test_multilabel_confusion_matrix_multilabels8"".   4&4rc Cstdddgdddgdddgg}tdddgdddgdddgg}tjtddt||ddgdW5QRXtjtdd,t||dddgddd gdd d ggdW5QRXd }tjt|dt||d gd W5QRXd }tjt|dt||dgd W5QRXtjtdd tdddgdddgddW5QRXd}tjt|d0tdddgdddggdddgdddggW5QRXdS)Nr2rzinconsistent numbers of samplesrr,rzshould be a 1d arrayrrrz%All labels must be in \[0, n labels\)rrzSamplewise metricsTrz'multiclass-multioutput is not supported)r7rrrrr()rGrFrrHrHrI'test_multilabel_confusion_matrix_errorss*""$rz%normalize, cm_dtype, expected_results)truerTUU?)predrr)allrgeq?)Nrr,cCsPdddgd}tttdddg}t|||d}t|||jj|ksLtdS)Nrr2r,r normalize)listrrrrdtypekindra)rZcm_dtypeZexpected_resultsy_testrFrrHrHrItest_confusion_matrix_normalize2s  rc CsTddddddddg}ddddddddg}tjtddt||ddW5QRXdS)Nrr2znormalize must be one ofrTr)rrrr)rrFrHrHrI,test_confusion_matrix_normalize_wrong_optionCsrc Csddddddddg}ddddddddg}t||dd}|tdksLtt tdtt||dd}W5QRX|tdkstt tdtt||ddW5QRXdS) Nrr2rr@errorrr) rsumrrrarxry simplefilterRuntimeWarning)rrFZcm_trueZcm_predrHrHrI,test_confusion_matrix_normalize_single_classJs    r zparams, warn_msgrGrFz2samples of only one class were seen during testingz:positive_likelihood_ratio ill-defined and being set to nanz+no samples predicted for the positive classz:negative_likelihood_ratio ill-defined and being set to nanz@no samples of the positive class were present in the testing setc Cs(tjt|dtf|W5QRXdSNr)rrrr)paramsZwarn_msgrHrHrItest_likelihood_ratios_warnings]s1rzparams, err_msgr,zeclass_likelihood_ratios only supports binary classification problems, got targets of type: multiclassc Cs(tjt|dtf|W5QRXdSr)rrrr)rrrHrHrItest_likelihood_ratios_errorssrcCstdgddgd}tdgddgddgd}t||\}}t|dt|d t||\}}t|tjdt|tdd d td gd dgd}t|||d\}}t|dt|ddS)Nr2rrrr,rrg?g_B{ %?g-q=)ZrtolrrlrrgUUUUUU@gqq?)r7rrrrrmr)rGrFposnegrrHrHrItest_likelihood_ratioss$   rcCstdgddgd}tdgddgddgddgd}t||}t|dd d |t||ksrtt|d gd }t|d gd }t||ddgd |ksttt||dtdgddgdd gd}tdgddgdd gd}tt||dd d tdgddgdd gd}tdgddgdd gd}tt||dd d tt||dddd d tt||dddd d dS)Nr(r2<rUr2gʡE?rrr,rrr.,4 g??g+?r/weightsg_vO?Z quadraticg#?)r7rrrraappend)rrkapparHrHrItest_cohen_kappas*. $$$$ r#cCs4tdgdgdksttddgddgdks0tdS)Nrr2rl)r"rarHrHrHrItest_matthews_corrcoef_nansr$cCsNtjd}|jdddd}|jdddd}tt||t||dddS)Nrr,rUrrr2r)r7r;r<rrr"Zcorrcoef)rErGrFrHrHrI-test_matthews_corrcoef_against_numpy_corrcoefs r&c stjd}|jdddd}|jdddd}|d}t|||dttfddtD}tfddtD}tfd dtD}|t ||}t |||d}t ||d dS) Nrr,rUrrc sXg|]P}tD]B}tD]4}||f||f||f||fqqqSrHrange)rkmlCNrHrIrs   z9test_matthews_corrcoef_against_jurman..cs@g|]8ddftfddtDqS)Ncs.g|]&}tD]}|kr||fqqSrHr'rrgr-r.r)rHrIrs  Dtest_matthews_corrcoef_against_jurman...r r7r(rr,r)rIrscs@g|]8ddftfddtDqS)Ncs.g|]&}tD]}|kr||fqqSrHr'r/r1rHrIr s  r2r3r4r,r5rIrsr) r7r;r<rrrr_r r(rr"r) rErGrFrZcov_ytypZcov_ytytZcov_ypypZ mcc_jurmanZmcc_oursrHr,rI%test_matthews_corrcoef_against_jurmans0     r6cCsftjd}dd|jddddD}tt||ddd|D}tt||d t|d d gd }t|d d }tt||d ttddddgddddgd tt|d gt|d ddddddddddddddddddddg}ddddddddddddddddddddg}tt||d dgddgd}t t tt|||dd W5QRXdS)NrcSsg|]}|dkrdndqS)rrrrsrHrrrHrHrIrsz*test_matthews_corrcoef..r,rUrrcSsg|]}|dkrdndqS)rrrsrHr7rHrHrIrsrrrrsrrlr2rr) r7r;r<rrr"r wherer_rrra)rErGZ y_true_invZ y_true_inv2y_1y_2maskrHrHrItest_matthews_corrcoefs   ,, r<c stjd}tdd}fdd|jd|ddD}tt||dddd d d d g}d d ddd d g}tt||d ddd d d d g}d d ddddg}tt||d td dd d g}dddg}tt||ddddg}dd d g}tt||ddd d dd d dd d g }d d d d d d dddg }tt||dddd d d g}d d ddd g}d d d d dg}tt|||ddddd d g}ddd d g}d d ddg}tt|||dddS)Nrrrrcsg|]}t|qSrH)chrr7Zord_arHrIr;sz5test_matthews_corrcoef_multiclass..rUrrr2r,giirrlrr)r7r;r<ordrrr"r) rE n_classesrGZ y_pred_badZ y_pred_minrFr9r:rrHr>rI!test_matthews_corrcoef_multiclass7sD          rAn_pointsdi'cstjddd}fdd}tddg|}tt||dtdddg|}tt||d||\}}tt||dtt|||||dS) Ni3c Ssxt||}|d}|d}|d}t|}|||}|||}||||} ||d|d|} | t| S)Nr2r2)r2rr%r2)rr_r7r) rGrFZ conf_matrixZtrue_posZ false_posZ false_negrBZpos_rateZactivityZ mcc_numeratorZmcc_denominatorrHrHrImcc_safets   z1test_matthews_corrcoef_overflow..mcc_safecs8|}|d|d}|dk}|dk}||fS)N皙?r)Z random_sample)rBZx_trueZx_predrGrFrErHrI random_yss  z2test_matthews_corrcoef_overflow..random_ysrlrr)r7r;r<repeatrr")rBrErHZarrrGrFrHrGrItest_matthews_corrcoef_overflowos   rJc CsRtdd\}}}t||dd\}}}}t|dddgdt|dd d gdt|d d d gdt|dddgt||ddd}t|ddt||dd}t|ddt||dd} t| ddt||dd}t|ddt||dd}t|ddt||dd} t| ddt||dd}t|ddt||dd}t|ddt||dd} t| ddtt t||ddW5QRXtt t||ddW5QRXtt t||ddW5QRXtt t ||dddW5QRXt||dddgdd\}}}}t|dddgdt|dd d gdt|d d d gdt|dddgdS)!NFrr(\?gQ?gzG?r,gHzG?g ףp= ?rSgQ?g333333?rrLrRrUr2rrg(\?rrgRQ?rgGz?rrrrrrg= ףp=?r) rJr#rrr$r%rrrrr) rGrFrfrDrrrrrrrHrHrI)test_precision_recall_f1_score_multiclasssT             rMrrrrrcCstddddgg}tddddgg}t||ddddgg|d\}}}}t|dt|dt|d|dkrt|ddddgdS)Nr2rrr,)r\warn_forr)r7rr#r)rrGrFrDrrrrHrHrI;test_precision_refcall_f1_score_multilabel_unordered_labelss    rOc Cstdddddddddddddddg}tdddddddddddddddg}t||dd\}}}}t||dd\}}}}|t|kst|t|kst|t|kstt||dd\}}}}t|} |tj|| dkst|tj|| dkst|tj|| dkstdS)Nrr2rrrr)r7rr#rraZbincountr) rGrFrrrrfrDrrrPrHrHrI.test_precision_recall_f1_score_binary_averageds(( rPc Cstjdd}zrtddddddg}tddddddg}tt||ddddtt||ddddtt||ddddW5tjf|XdS) Nraise)rrr2r,rrrl)r7Zseterrrrr$r%r)Zold_error_settingsrGrFrHrHrItest_zero_precision_recalls rRcCstdd\}}}t||ddgd}t|ddgddggt||d dgd}t|d d gd dggt|d}t||d |gd}t|d dgddggdS) NFrrr2rrrrr,rrL)rJrrr7max)rGrFrfrZ extra_labelrHrHrI.test_confusion_matrix_multiclass_subset_labelssrTzlabels, err_msgz,'labels' should contains at least one label.rrz.At least one label specified must be in y_truez empty listzunknown labels)idsc Cs<tdd\}}}tjt|dt|||dW5QRXdS)NFrrr)rJrrrr)r\rrGrFrfrHrHrItest_confusion_matrix_errors rVr\Noner? multiclasscCs>|r t|nd}tj||ftd}tgg|d}t||dS)Nrrr)r_r7rr:rr)r\Zexpected_n_classesexpectedrrHrHrI*test_confusion_matrix_on_zero_length_inputsr[cCs>dddg}tt|}t||}|jtjks2ttjtjtj fD],}t|||j |ddd}|jtjksBtqBtj tj dt fD],}t|||j |ddd}|jtj kstqtjt|dtjd}t|||d}|ddkst|d d ksttjt|d tjd}t|||d}|dd ks(t|d d ks:tdS) Nrr2F)copyrlrYrrrDll)r7Zonesr_rrZint64raZbool_Zint32Zuint64ZastypeZfloat32Zfloat64objectfullZuint32)rAZweightrrrHrHrItest_confusion_matrix_dtype's$  racCsLt}t|dd\}}}d}t||tt|j|jd}||ksHtdS)NFrKa| precision recall f1-score support setosa 0.83 0.79 0.81 24 versicolor 0.33 0.10 0.15 31 virginica 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 r\r] rr3rJrr7r8r_r]rarerGrFrfrgrhrHrHrI%test_classification_report_multiclassBs rec CsLdddddddddg dddddddddg }}d}t||}||ksHtdS)Nrr2r,a| precision recall f1-score support 0 0.33 0.33 0.33 3 1 0.33 0.33 0.33 3 2 0.33 0.33 0.33 3 accuracy 0.33 9 macro avg 0.33 0.33 0.33 9 weighted avg 0.33 0.33 0.33 9 )rra)rGrFrgrhrHrHrI.test_classification_report_multiclass_balanced\s. rfcCs8t}t|dd\}}}d}t||}||ks4tdS)NFrKa| precision recall f1-score support 0 0.83 0.79 0.81 24 1 0.33 0.10 0.15 31 2 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 )rr3rJrrardrHrHrI:test_classification_report_multiclass_with_label_detectionns  rgcCsNt}t|dd\}}}d}t||tt|j|jdd}||ksJtdS)NFrKa| precision recall f1-score support setosa 0.82609 0.79167 0.80851 24 versicolor 0.33333 0.09677 0.15000 31 virginica 0.41860 0.90000 0.57143 20 accuracy 0.53333 75 macro avg 0.52601 0.59615 0.50998 75 weighted avg 0.51375 0.53333 0.47310 75 r)r\r]digitsrcrdrHrHrI1test_classification_report_multiclass_with_digitss ricCsztdd\}}}tdddg|}tdddg|}d}t||}||ksRtd}t||dd d gd }||ksvtdS) NFrbluegreenreda| precision recall f1-score support blue 0.83 0.79 0.81 24 green 0.33 0.10 0.15 31 red 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 a| precision recall f1-score support a 0.83 0.79 0.81 24 b 0.33 0.10 0.15 31 c 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 rrrsrtr]rJr7rrra)rGrFrfrgrhrHrHrI7test_classification_report_multiclass_with_string_labels   rocCsNtdd\}}}tdddg}||}||}d}t||}||ksJtdS)NFrublue¢ugreen¢ured¢u precision recall f1-score support blue¢ 0.83 0.79 0.81 24 green¢ 0.33 0.10 0.15 31 red¢ 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 rnrGrFrfr\rgrhrHrHrI8test_classification_report_multiclass_with_unicode_labels rqcCsNtdd\}}}tdddg}||}||}d}t||}||ksJtdS)NFrrjZgreengreengreengreengreenrla precision recall f1-score support blue 0.83 0.79 0.81 24 greengreengreengreengreen 0.33 0.10 0.15 31 red 0.42 0.90 0.57 20 accuracy 0.53 75 macro avg 0.53 0.60 0.51 75 weighted avg 0.51 0.53 0.47 75 rnrprHrHrIpos_label=2 is not a valid label. It should be one of \[0, 1\]rr?r,rrTarget is multilabel-indicator but average='binary'. Please choose another average setting, one of \[None, 'micro', 'macro', 'weighted', 'samples'\].rTarget is multiclass but average='binary'. Please choose another average setting, one of \[None, 'micro', 'macro', 'weighted'\].rzJSamplewise metrics are not available outside of multilabel classification.rzNote that pos_label \(set to 3\) is ignored when average != 'binary' \(got 'micro'\). You may use labels=\[pos_label\] to specify a single positive class.rr)r7rrrrr rr)rGrFrZmsg1msg2msg3r}rHrHrItest_jaccard_score_validationFs0rc Cs$tdddgdddgg}tdddgdddgg}t||dddksJtt||dddks`tt||dddksvtt|t|dddkstt|t|dddkstt|t|jdddkstt|t|jdddksttdddgdddgg}tdddgdddgg}tt||dddtt||ddd tt||ddd tt||ddd gd d tt||ddd gd d tt||ddtd dd gtdddgdddgg}tdddgdddgg}tt||dddtt||dddd}t j t |dt||dgddW5QRXd}t j t |dt||dgddW5QRXd}t j t |d6ttddggtddggddd kstW5QRXd}t j t |dBttddgddggtddgddggddd kstW5QRXt|r tdS)Nrr2rrrrrrrg?r,rrrrrg?z Got 4 > 2rrrz Got -1 < 0rzXJaccard is ill-defined and being set to 0.0 in labels with no true or predicted samples.zXJaccard is ill-defined and being set to 0.0 in samples with no true or predicted labels.)r7rr rarrr6rrrrrrr*r)recwarnrrrGrFrrr}rHrHrItest_multilabel_jaccard_scorensn $ rc Csbddddddddg}ddddddddg}dddg}t}||||}||}tt||}tt||}ddgddgddgdgdgdgdg} ddgddgddgdgdgdgdg} dD]2} t| | D]"\} } t|| | d|| | dqqtddgddgddgg}tddgddgddgg}t t||d d dksFt W5QRXt |r^t dS) NZantcatZbirdrr2r,)rrrNrrr) r r=Z transformrr ziprr7rrrar)rrGrFr\ZlbrrZmulti_jaccard_scoreZbin_jaccard_scoreZmulti_labels_listZbin_labels_listrZm_labelZb_labelrHrHrItest_multiclass_jaccard_scores:       $  "rc Cstdgdgdddkstd}tjt|d$tddgddgdddksLtW5QRXtdgdgdddd ksrttdddddg}tdddddg}tt||ddd tt||ddd d t|rtdS) Nr2rr?rrlzOJaccard is ill-defined and being set to 0.0 due to no true or predicted samplesrrrrrr) r rarrr*r7rrr)rr}rGrFrHrHrI!test_average_binary_jaccard_scores(rc Csxtdddgdddgg}tdddgdddgg}d}tjt|d(t||ddd}|tdksjtW5QRXdS) Nr2rzJaccard is ill-defined and being set to 0.0 in samples with no true or predicted labels. Use `zero_division` parameter to control this behavior.rrrqrrprl)r7rrrr*r rra)rGrFr}scorerHrHrI(test_jaccard_score_zero_division_warningsrzzero_division, expected_scorer])r2rc Csztdddgdddgg}tdddgdddgg}t"tdtt||d|d}W5QRX|t|ksvt dS)Nr2rr rr) r7rrxryr r*r rrra)rpZexpected_scorerGrFrrHrHrI*test_jaccard_score_zero_division_set_values  rcCsftddddgddddgddddgg}tddddgddddgddddgg}t||dd\}}}}t|ddddgdt|ddddgdt|ddddgdt|ddddgdt||ddd }|}t|dd ddgdt||d d\}}}}t|d t|dt|d |dks ttt||dd d t|t||dd\}}}}t|dt|dt|d|dks~ttt||ddd d||d||t||dd\}}}}t|d t|dt|d |dksttt||ddd tj||dt||dd\}}}}t|dt|dt|d|dksLttt||ddd ddS)Nr2rrrlrrr,rrrrKrg?竪?rrrrrr r7rr#rrrrarrrGrFrDrrrf2rPrHrHrI+test_precision_recall_f1_score_multilabel_1sV((             rcCshtddddgddddgddddgg}tddddgddddgddddgg}t||dd\}}}}t|ddddgdt|ddddgdt|ddddgdt|ddddgdt||ddd }|}t|dd ddgdt||d d\}}}}t|d t|d t|d |dks ttt||dd d d ||d||t||dd\}}}}t|d t|dt|d|dksttt||ddd t|t||dd\}}}}t|dt|d t|d|dksttt||ddd tj||dt||dd\}}}}t|dt|dt|d|dksLttt||ddd dddS)Nr2rrrlrr,rgQ?r皙?r?rrrg?rzrrQrrg&S?rrrHrHrI+test_precision_recall_f1_score_multilabel_2Vs^((             rc Cstddddgddddgddddgg}tddddgddddgddddgg}|dkr\dnd}t||d|d\}}}}t||dddgdt|ddd|gdt|ddddgdt|ddddgdt||dd|d }|}t|dd ddgdt||d |d\}}}}t|d|d t|d |d t|d|dksFttt||dd dt|t||d|d\}}}}t|dt|dt|d|dksttt||dd|d d||d ||t||d|d\}}}}t||dkrdndt|dt|d|dks$ttt||dd|d tj||dt||dd\}}}}t|dt|dt|d|dksttt||dd|d dddS)Nrr2rrlrr,rrrrrprrr?rrrrTrrrg?rrrrQgZd;O?r) rprGrFrDrrrrrPrHrHrI7test_precision_recall_f1_score_with_an_empty_predictions((          rrc Cstd}t|}tt|||||d\}}}}tt|||||d} t|}t||t||t|||dksvtt| t|dS)NrUrrrrpr) r7rr{rr#rrcrra) rrrprGrFrDrrrfbetarHrHrI"test_precision_recall_f1_no_labelss0       rc Cstd}t|}t}tt||||dd\}}}}W5QRXt|dt|dt|d|dksptttt |||dd}W5QRXt|ddS)NrrrLr) r7rr{r#rrr*rrar) rrGrFfuncrDrrrrrHrHrI1test_precision_recall_f1_no_labels_check_warningss   "     rcCstd}t|}tt||dd|d\}}}}tt||dd|d}t|}t||||gdt||||gdt||||gdt|dddgdt||||gddS)Nrrrrr,r)r7rr{rr#rrcr)rprGrFrDrrrrrHrHrI/test_precision_recall_f1_no_labels_average_nones0  rc Cstd}t|}ttt||ddd\}}}}W5QRXt|dddgdt|dddgdt|dddgdt|dddgdttt||ddd}W5QRXt|dddgddS)Nrr2rLrr,r) r7rr{rrr*r#rr)rGrFrDrrrrrHrHrI4test_precision_recall_f1_no_labels_average_none_warn<s    rc Cstt}}dD]t}d}tj||d |dddgdddg|dW5QRXd}tj||d |dddgdddg|dW5QRXqd }tj||d8|tddgddggtddgddggd dW5QRXd }tj||d8|tddgddggtddgddggd dW5QRXd }tj||d8|tddgddggtddgddggd dW5QRXd}tj||d8|tddgddggtddgddggd dW5QRXd }tj||d|ddgddgddW5QRXd}tj||d|ddgddgddW5QRXtjdd^}tdtddgddgddd}t | j |ksvt d }t | j |kst W5QRXdS)NNrrzPrecision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.rrr2r,rzRecall and F-score are ill-defined and being set to 0.0 in labels with no true samples. Use `zero_division` parameter to control this behavior.zPrecision and F-score are ill-defined and being set to 0.0 in samples with no predicted labels. Use `zero_division` parameter to control this behavior.rzRecall and F-score are ill-defined and being set to 0.0 in samples with no true labels. Use `zero_division` parameter to control this behavior.zPrecision and F-score are ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.rzRecall and F-score are ill-defined and being set to 0.0 due to no true samples. Use `zero_division` parameter to control this behavior.rr?Trvalways) r#r*rrr7rrxryr rzpopr{ra)rr}rr}rwrHrHrItest_prf_warningsXsV $&<<<<   rc Cst}dD]@}t|dddgdddg||dt|dddgdddg||dqt|tddgddggtddgddggd|dt|tddgddggtddgddggd|dt|tddgddggtddgddggd|dt|tddgddggtddgddggd|dt|ddgddgd |dt|ddgddgd |dtjd d :}td tddgddgd |dt|dkstW5QRXdS) Nrrr2r,rrrrr?Trvr) r#rr7rrxryr r_ra)rprrrwrHrHrI)test_prf_no_warnings_if_zero_division_sets   rc Cstttddgddggtddgddggd|dtjdd}tdttddgddggtddgddggd|d|dkrt|j d kst nt |dkst tddgddg|dkrt|j d kst W5QRXdS) Nr2rrrTrvrrqzzRecall is ill-defined and being set to 0.0 due to no true samples. Use `zero_division` parameter to control this behavior.) rr%r7rrxryr rzrr{rar_rprwrHrHrItest_recall_warningss6   rc Cstjdd}tdttddgddggtddgddggd|d|dkrlt|jd ks|t nt |dks|t tddgddg|dkrt|jd kst W5QRXt ttddgddggtddgddggd|ddS) NTrvrr2rrrrqzPrecision is ill-defined and being set to 0.0 due to no predicted samples. Use `zero_division` parameter to control this behavior.) rxryr r$r7rrzrr{rar_rrrHrHrItest_precision_warnings#s6   rc Cs.tjdd}tdtttddfD]}|tddgddggtddgddggd|d t|dksrt |tddgddggtddgddggd|d t|dkst |tddgddggtddgddggd|d |d krt | j d kst q*t|dks*t q*W5QRXdS) NTrvrr,rr2rrrrqzF-score is ill-defined and being set to 0.0 due to no true nor predicted samples. Use `zero_division` parameter to control this behavior.) rxryr rrrr7rr_rarzrr{)rprwrrHrHrItest_fscore_warningsKs:    rc Csddddg}ddddg}d}tdddgdddgdddgg}tdddgdddgdddgg}d}|||f|||ffD]H\}}}tttttddfD](} tjt |d| ||W5QRXqqxdS) Nr2r,rrrrrr) r7rr$r%rrrrrr) Z y_true_mcZ y_pred_mcZmsg_mcZ y_true_indZ y_pred_indZmsg_indrGrFr}rjrHrHrI'test_prf_average_binary_data_non_binaryss$  "" rc*Cs~d}d}d}d}d}d}|tdddgdddgdddggf|tddgddgddggf|d d dgf|dddgf|d d d gf|td gd gdggf|tdgdgdggf|td gd gd ggf|tdd gdd gd d ggf|tddgddgddggfg }||f|||f|||f|||fd||fd||f|||fd||fd||fd||fd||fd||fd||fd||fd||fd||fd||fd||fd||fd||fd||fdi}t|d dD]\\}} \} } z||| f} Wn"tk r|| |f} YnX| dkrttt| | W5QRX|| krjd|| } tjt| dt| | W5QRXn>||||fkr>d|} tjt| dt| | W5QRXnt| | \}}}|| kst | dr|jdkst |jdkst n t |t | t |t | ttt| dd| W5QRXqddg} ddg} d}tjt|dt| | W5QRXdS)Nzmultilabel-indicatorrXr?Z continuouszmulticlass-multioutputzcontinuous-multioutputrr2r,rrlrrrrg?g@)rIz@Classification metrics can't handle a mix of {0} and {1} targetsrz{0} is not supportedZ multilabelZcsrr)r2r,)rr,r)r,)rr,zYou appear to be using a legacy multi-label data representation. Sequence of sequences are no longer supported; use a binary array or sparse matrix instead - the MultiLabelBinarizer transformer can convert to this format.) r7rrKeyErrorrrrr)formatra startswithrZsqueeze)ZINDZMCZBINZCNTZMMCZMCNZEXAMPLESZEXPECTEDZtype1rZtype2rrZrZ merged_typeZy1outZy2outr}rHrHrItest__check_targetss$          rcCs*ddg}ddg}t||ddks&tdS)Nrr2rrX)r)rarrHrHrIAtest__check_targets_multiclass_with_both_y_true_and_y_pred_binarysrcCsptddddg}tddddg}t||dks6ttdd d dg}tddddg}t||dksltdS) Nrr2g!rrg333333ӿ333333?rr,)r7rrrarG pred_decisionrHrHrItest_hinge_loss_binarys rc Cs6tddddgddddgddd dgdd ddgd d d d gddd dgg}tddddddg}td|dd|ddd|dd|ddd|dd|ddd|dd|ddd|dd|ddd|dd|ddg}tj|dd|dt|}t|||ks2tdS)N ףp= ?(\ſ(\GzgHzGgGz׿Q޿333333RQؿzGHzGHzGѿQ?rr2r,rrroutr7rcliprrra)rrG dummy_lossesdummy_hinge_lossrHrHrItest_hinge_loss_multiclass s,         rc Csptddddg}tddddgdd d d gd d ddgd d ddgg}d}tjt|dt||W5QRXdS)Nrr2r,gRQ?g rh?g(\gffffffrrrrrrrrzDPlease include all labels in y_true or pass labels as third argumentr)r7rrrrr)rGr error_messagerHrHrI:test_hinge_loss_multiclass_missing_labels_with_labels_none s     rc Cstdddddddg}tdddddddg}d}tjtt|dt||dW5QRXtddgddgddgddgddgddgddgg}dddg}d}tjtt|dt|||dW5QRXdS) Nr,r2rzThe shape of pred_decision cannot be 1d arraywith a multiclass target. pred_decision shape must be (n_samples, n_classes), that is (7, 3). Got: (7,)rrzThe shape of pred_decision is not consistent with the number of classes. With a multiclass target, pred_decision shape must be (n_samples, n_classes), that is (7, 3). Got: (7, 2))rGrr\)r7rrrrreescaper)rGrrr\rHrHrIt td d gd d gd dtj ks`t td d gd d gd dtj kst td d dgd d d gd d d gd d d ggd dd kst td d dgd ddgd d d gd d d ggd dtj kst d d dg}ddgddgddgg}t t t||W5QRXddddg}ddgddgddgddgg}t||}t|dddddg}ddgddgg}tddgddgg}d }t j t |d!t||W5QRXddgddgddgg}d"}t |t||ftt|ddd f }t||d dgd#}t||d ddg}dddgdddgdddgg}t||d dd$gd#}t|d%dddS)&NnoyesrrrSg{Gz?gGz?rrgMbP?g+?r2rr,rFffffff?rrTrg躕ʀ?Fg.L`@rr)repsrrhamspamL7?zly_true contains only one label \(2\). Please provide the true labels explicitly through the labels argument.rzBFound input variables with inconsistent numbers of samples: [3, 2]rrgPv0?)r7rr!rrZlogpmfrZasarrayrrarrrrlog) rGrFlossZ loss_truerZ error_strZ true_log_lossZcalculated_log_lossZy_score2rHrHrI test_log_loss s`& *    ""68      rcCs:tjddg|d}|}t||dd}t|s6tdS)zCheck the behaviour of `eps="auto"` that changes depending on the input array dtype. Non-regression test for: https://github.com/scikit-learn/scikit-learn/issues/24315 rr2rYautorN)r7rr\r!isfinitera)Z global_dtyperGrFrrHrHrItest_log_loss_eps_auto srcCs<tjddgtjd}|}t||dd}t|s8tdS)z2Check the behaviour of `eps="auto"` for np.float16rr2rYrrN)r7rZfloat16r\r!rra)rGrFrrHrHrItest_log_loss_eps_auto_float16 src Cstddddg}tddgddgddgddgg}ttfg}z"d d lm}m}|||fWntk rtYnX|D]2\}}||||}}t||} t | d d d qzdS)NrrrFrrrrrr)Series DataFramerrr) r7rrZpandasrrr! ImportErrorr!r) Zy_trZy_prtypesrrZ TrueInputTypeZ PredInputTyperGrFrrHrHrItest_log_loss_pandas_input s"   rc Cstddddddg}tddddddg}t||d t|}tt||d tt|||ttd|||ttd |d||tt t||ddW5QRXtt t||dW5QRXtt t||dW5QRXtddd dg}tdd d d g}d}tjt |dt||W5QRXttdgd gdttdgd gdttdgd gdttdgd gdddttdgd gddddS)Nrr2rrrSrrgffffff?r,rlrrrFzMOnly binary classification is supported. The type of the target is multiclassrrg{Gz?rZfoobar)r) r7rrZnormr_rr'rrr)rGrFZ true_scorerrHrHrItest_brier_score_loss s0   rc Cs8d}tjt|dtdddgdddgW5QRXdS)Nz%y_pred contains classes not in y_truerrr2)rrrrrrHrHrI#test_balanced_accuracy_score_unseen< srz y_true,y_predrrrsrtc Cst||dt|d}tt||}W5QRX|t|ksDtt||dd}t|t||d}|||d|kstdS)NrrT)adjustedrr2) r%r7uniquerrrrraZ full_like)rGrFZ macro_recallZbalancedrZchancerHrHrItest_balanced_accuracy_scoreB s r)NF) functoolsr itertoolsrrrrxrZnumpyr7ZscipyrZ scipy.statsrrZsklearnrr Zsklearn.datasetsr Zsklearn.preprocessingr r Zsklearn.utils.validationr Zsklearn.utils._testingrrrrrrZsklearn.utils._mockingrZsklearn.metricsrrrrrrrrrrrr r!r"r#r$r%r&r'r(Zsklearn.metrics._classificationr)Zsklearn.exceptionsr*Zscipy.spatial.distancer+r|rJrkromarkZ parametrizer~rrrrrrrrrrrrrrrrrr rrrrr#r$r&r6r<rArJrMrOrPrRrTrVr[rarerfrgrirorqrrrvrwrxryr~rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrHrHrHrIs                                         0B "  -      #)     -   '#8 !:    #  (M%  B @ L  !c @ & ' 'b  N  # &&&