jo4ddlmZmZmZmZmZddlZddlZddl Z ddl Z ddl mZmZddlmZmZGddZdZdZd ZGd d eZd Zd Zd ZGddeZdded edefdZdS))ListUnionOptionalSetTupleN) AnnotationTimeline)DetailsMetricComponentsc eZdZdZedefdZedefdZfdZ dZ dZ e dZ dd eeefd eeefd edeefdZddedejfdZdZdZdedeeeffdZdZd eeefd eeefdefdZdefdZddedeeeeefffdZ xZ!S) BaseMetricz :class:`BaseMetric` is the base class for most pyannote evaluation metrics. Attributes ---------- name : str Human-readable name of the metric (eg. 'diarization error rate') returnc0t|jdz)Nz\ is missing a 'metric_name' class method. It should return the name of the metric as string.NotImplementedError__name__clss x/home/lighthouse/badge-speaker-benchmark/.benchmark-venv-community/lib/python3.11/site-packages/pyannote/metrics/base.py metric_namezBaseMetric.metric_name1s%! LP P   c0t|jdz)Nzh is missing a 'metric_components' class method. It should return the list of names of metric components.rrs rmetric_componentszBaseMetric.metric_components8s%! LV V   rc tt||j|_t |j|_| dSN) superr __init__ __class__r metric_name_setr components_reset)selfkwargsrs rrzBaseMetric.__init__?sd j$((*** N6688%()I)I)K)K%L%L rc$d|jDS)Nci|]}|dS)).0values r z.BaseMetric.init_components..Fs999us999r)r!r#s rinit_componentszBaseMetric.init_componentsEs99(89999rczt|_t|_|jD] }d|j|< dS)z.Reset accumulated components and metric valuesr'N)dict accumulated_listresults_r!)r#r*s rr"zBaseMetric.resetHsE%)VV"ff % + +E'*D e $ $ + +rc*|S)z Metric name.)rr,s rnamezBaseMetric.nameOs!!!rFN reference hypothesisdetaileduric $|j||fi|}||||j<|pt|dd}|j||f|jD]}|j|xx||z cc<|r|S||jS)aCompute metric value and accumulate components Parameters ---------- reference : type depends on the metric Manual `reference` hypothesis : type depends on the metric Evaluated `hypothesis` uri : optional Override uri. detailed : bool, optional By default (False), return metric value only. Set `detailed` to True to return dictionary where keys are components names and values are component values Returns ------- value : float (if `detailed` is False) Metric value components : dict (if `detailed` is True) `components` updated with metric value r8NA)compute_componentscompute_metricrgetattrr2appendr!r0)r#r5r6r7r8r$ componentsr4s r__call__zBaseMetric.__call__Ws6-T,Y MMfMM )-(;(;J(G(G 4$%4WYt44 c:.///$ 8 8D  d # # #z$'7 7 # # # #   $+,,rdisplaychg}g}d|v}|jD]\}}i}|r|d}|D]W\} } | |jkr d| z|| df<| dkr| || df<)| || df<|r%|dkrd| z|z || df<Ftj|| df<X||||i}|j}|r|d}|D]W\} } | |jkr d| z|| df<| dkr| || df<)| || df<|r%|dkrd| z|z || df<Ftj|| df<Xdt|z||jdf<|||dtj |} || d<| d} tj | j| _| |jg|z} |r't| dd d d | S) akEvaluation report Parameters ---------- display : bool, optional Set to True to print the report to stdout. Returns ------- report : pandas.DataFrame Dataframe with one column per metric component, one row per evaluated item, and one final row for accumulated results. totald%rTOTALitemTFrightc,d|SNz{0:.2f}formatfs rz#BaseMetric.report..s9+;+;A+>+>r)indexsparsifyjustify float_format)rr2itemsr4npnanr>r0abspd DataFrame set_index MultiIndex from_tuplescolumnsprint to_string) r#rAreporturispercentr8r?rowrCkeyr*dfs rrazBaseMetric.reportsT33555#}  OCC ,"7+(..00 3 3 U$)##$'%KCSMMG^^#(CRLL#(CRL3 199,/%K%,?CSMM,.FCSM MM#    KK    &  (w'E$**,, / /JCdi #e CH $CG $CG /qyy(+e e(;CH (*CH !CIIoDIsN c G \& ! !6 \\& ! !]..rz::  d44666 7    "#!>!>     rc^|d}|ddS)NF)rAc,d|SrKrLrNs rrPz$BaseMetric.__str__..s93C3CA3F3Fr)rRrT)rar`)r#ras r__str__zBaseMetric.__str__s;U++)F)F    rc6||jS)z0Compute metric value from accumulated components)r<r0r,s r__abs__zBaseMetric.__abs__s""4#4555r componentcn|tdddkrt|jS|j|S)aGet value of accumulated `component`. Parameters ---------- component : str Name of `component` Returns ------- value : type depends on the metric Value of accumulated `component` N)slicer/r0)r#rls r __getitem__zBaseMetric.__getitem__s: dD$// / /)** *$Y/ /rc#0K|jD] \}}||fV dS)z*Iterator over the accumulated (uri, value)N)r2)r#r8rls r__iter__zBaseMetric.__iter__s8"m ! !NCy.  ! !rc :t|jjdz)aCompute metric components Parameters ---------- reference : type depends on the metric Manual `reference` hypothesis : same as `reference` Evaluated `hypothesis` Returns ------- components : dict Dictionary where keys are component names and values are component values z is missing a 'compute_components' method.It should return a dictionary where keys are component names and values are component values.rrr)r#r5r6r$s rr;zBaseMetric.compute_componentss*(" N #'I I   rr?c:t|jjdz)aACompute metric value from computed `components` Parameters ---------- components : dict Dictionary where keys are components names and values are component values Returns ------- value : type depends on the metric Metric value z is missing a 'compute_metric' method. It should return the actual value of the metric based on the precomputed component dictionary given as input.rs)r#r?s rr<zBaseMetric.compute_metrics*" N #'` `   r?alphac.fdjD}t|dkrtdt|dkr&tjd|dx}x}}|||ffSt j||dS)aPCompute confidence interval on accumulated metric values Parameters ---------- alpha : float, optional Probability that the returned confidence interval contains the true metric value. Returns ------- (center, (lower, upper)) with center the mean of the conditional pdf of the metric value and (lower, upper) is a confidence interval centered on the median, containing the estimate to a probability alpha. See Also: --------- scipy.stats.bayes_mvs c0g|]\}}|jSr()r)r)_rr#s r z2BaseMetric.confidence_interval..?s%AAA41a!D%&AAArrzFPlease evaluate a bunch of files before computing confidence interval.zCCannot compute a reliable confidence interval out of just one file.)rv)r2len ValueErrorwarningswarnscipystats bayes_mvs)r#rvvaluescenterloweruppers` rconfidence_intervalzBaseMetric.confidence_interval(s.BAAA4=AAA v;;!  eff f [[A   M_ ` ` `%+AY .F .UUE5>) );((u(==a@ @r)FN)F)ru)"r __module__ __qualname____doc__ classmethodstrrr rrr-r"propertyr4rr rboolrr@rYrZrarirkfloatr rorqr;r<rr __classcell__)rs@rr r 'sM C   [  "2   [  :::+++""X"?C+-+-%*(<"=+-"8Z#78+-+-.6sm+-+-+-+-ZRRdRr|RRRRh   6660S0U5'>-B0000&!!!  &+Hj,@&A ',Xz-A'B )0    4     ("A"A"AUE%,//0"A"A"A"A"A"A"A"Arr precisionz # retrievedz# relevant retrievedcTeZdZdZedZedefdZdede fdZ dS) PrecisionaU :class:`Precision` is a base class for precision-like evaluation metrics. It defines two components '# retrieved' and '# relevant retrieved' and the compute_metric() method to compute the actual precision: Precision = # retrieved / # relevant retrieved Inheriting classes must implement compute_components(). ctSr)PRECISION_NAMErs rrzPrecision.metric_name^srrcttgSr)PRECISION_RETRIEVEDPRECISION_RELEVANT_RETRIEVEDrs rrzPrecision.metric_componentsbs#%ABBrr?cz|t}|t}|dkr|dkrdStd||z S)z#Compute precision from `components`r'r?rF)rrr~r#r? numerator denominators rr<zPrecision.compute_metricfsH;<  !45 #  A~~s nn${* *rN rrrrrrr rr rr<r(rrrrRs  [C"2CCC[C + +U + + + + + +rrrecallz # relevantcTeZdZdZedZedefdZdede fdZ dS)RecallaG :class:`Recall` is a base class for recall-like evaluation metrics. It defines two components '# relevant' and '# relevant retrieved' and the compute_metric() method to compute the actual recall: Recall = # relevant retrieved / # relevant Inheriting classes must implement compute_components(). ctSr) RECALL_NAMErs rrzRecall.metric_namesrrcttgSr)RECALL_RELEVANTRECALL_RELEVANT_RETRIEVEDrs rrzRecall.metric_componentss!:;;rr?cz|t}|t}|dkr|dkrdStd||z S)z Compute recall from `components`r'rrrF)rrr~rs rr<zRecall.compute_metricsG89  1 #  A~~s nn${* *rNrr(rrrrxs  [<"2<<<[< + +U + + + + + +rrrrcL||zdkrdSd||zz|z|z||z|z|zz S)uCompute f-measure f-measure is defined as follows: F(P, R, b) = (1+b²).P.R / (b².P + R) where P is `precision`, R is `recall` and b is `beta` r'rr|r()rrbetas r f_measurersD6S  q t Oy (6 1TD[95Lv5U VVr)r)typingrrrrrrnumpyrVpandasrY scipy.statsr pyannote.corerr pyannote.metrics.typesr r r rrrrrrrrrrr(rrrs854444444444444........<<<<<<<<cAcAcAcAcAcAcAcAL #5+++++ +++B 2+++++Z+++B W W W WE W W W W W Wr