j'GdZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZd d lmZGddZdZdS)ah Pipeline Usage: pyannote-pipeline train [options] [(--forever | --iterations=)] pyannote-pipeline best [options] pyannote-pipeline apply [options] pyannote-pipeline -h | --help pyannote-pipeline --version Common options: Experimental protocol (e.g. "Etape.SpeakerDiarization.TV") --registry= Path to, comma-separated, database configuration files. [default: ~/.pyannote/db.yml] --subset= Set subset. Defaults to 'development' in "train" mode, and to 'test' in "apply" mode. "train" mode: Set experiment root directory. This script expects a configuration file called "config.yml" to live in this directory. See "Configuration file" section below for more details. --iterations= Number of iterations. [default: 1] --forever Iterate forever. --sampler= Choose sampler between RandomSampler or TPESampler [default: TPESampler]. --pruner= Choose pruner between MedianPruner or SuccessiveHalvingPruner. Defaults to no pruning. --pretrained= Use parameters in existing training directory to bootstrap the optimization process. In practice, this will simply run a first trial with this set of parameters. --average-case Optimize for average case instead of worst case. "apply" mode: Path to the directory containing trained hyper- parameters (i.e. the output of "train" mode). --use-filter Apply pipeline only to files that pass the filter. Configuration file: The configuration of each experiment is described in a file called /config.yml that describes the pipeline. ................... /config.yml ................... pipeline: name: Yin2018 params: sad: tutorials/pipeline/sad scd: tutorials/pipeline/scd emb: tutorials/pipeline/emb metric: angular # preprocessors can be used to automatically add keys into # each (dict) file obtained from pyannote.database protocols. preprocessors: audio: ~/.pyannote/db.yml # load template from YAML file video: ~/videos/{uri}.mp4 # define template directly # filters can be used to filter out some files from the protocol # (e.g. to only keep files with a specific number of speakers) filters: pyannote.audio.utils.protocol.FilterByNumberOfSpeakers: num_speakers: 2 # one can freeze some hyper-parameters if needed (e.g. when # only part of the pipeline needs to be updated) freeze: speech_turn_segmentation: speech_activity_detection: onset: 0.5 offset: 0.5 # pyannote.audio pipelines will run on CPU by default. # use `device` key to send it to GPU. device: cuda ................................................................... "train" mode: Tune the pipeline hyper-parameters /..yml "best" mode: Display current best loss and corresponding hyper-paramters. "apply" mode Apply the pipeline (with best set of hyper-parameters) N)Optional)Path)docopt)tqdm)datetime) FileFinder)registry) get_annotated)get_class_by_name) OptimizerceZdZdZdZdZdZeddede ddfd Z dd ede ffd Z dde de e de edede e de e de fdZdde de fdZ dde dede e de fdZxZS) ExperimentzPipeline experiment Parameters ---------- experiment_dir : `Path` Experiment root directory. training : `bool`, optional Switch to training mode z{experiment_dir}/config.ymlz*{experiment_dir}/train/{protocol}.{subset}z{train_dir}/apply/{date}F train_dirtrainingreturnc|jd}|||}|dz }tjtj||_|j||S)a6Load pipeline from train directory Parameters ---------- train_dir : `Path` Path to train directory training : `bool`, optional Switch to training mode. Returns ------- xp : `Experiment` Pipeline experiment. r r params.yml) parentsr fromtimestampospathgetmtimemtime_ pipeline_ load_params)clsrrexperiment_dirxp params_ymls /home/lighthouse/badge-speaker-benchmark/.benchmark-venv-community/lib/python3.11/site-packages/pyannote/pipeline/experiment.pyfrom_train_dirzExperiment.from_train_dirsl #*1- S( 3 3 3- *27+;+;J+G+GHH    ,,, rc t||_|j|j}t |d5}t j|t j|_ dddn #1swxYwYi}|j di D]\}}t|tr7t|dd}|di| di||<Q t| ||<f#t $r} |} | ||<Yd} ~ |d} ~ wwxYw||_g|j d i D]/\}} t|}|di| 0d t&ffd } | |_|j d d} t| d}|di|j d  di|_d|j vr'|j d} |j| d|j vr@ddl}||j d}|j|dSdS)N)rrLoader preprocessorsnamezpyannote.pipeline)default_module_nameparams) database_ymlfiltersrc<tfdDS)Nc3.K|]}|VdS)N).0fis r" z;Experiment.__init__..all_filters..s+--qqtt------r$)all)r4r.s`r" all_filtersz(Experiment.__init__..all_filterss'----W----- -r$pipelinezpyannote.pipeline.blocksfreezedevicerr1)super__init__r CONFIG_YMLformatopenyamlload SafeLoaderconfig_getitems isinstancedictr rFileNotFoundErrorpreprocessors_appendboolfilters_rr9torchr:to)selfrr config_ymlfpr)key preprocessorKlassetemplater,r7 pipeline_namerMr:r. __class__s @r"r<zExperiment.__init__sB ,_++4;N+OO *c " " Ab9R@@@DL A A A A A A A A A A A A A A A !%!1!1/2!F!F!L!L!N!N . . C,-- ) (>Q&+U%L%L\-=-=h-K-K%L%L c" .&0\%J%J%J c""$ . . .(%- c""""""  . ,<++Ir::@@BB , ,KC%c**E NN55??6?? + + + + .d . . . . . .$  Z08 ! /I   LLj!9!=!=h!K!KLL t| # #\(+F N ! !& ) ) ) t| # # LLL\\$,x"899F N  f % % % % % $ #s*&B  BBD,, E6EE developmentNr protocol_namesubset pretrained n_iterationssamplerpruner average_casec t|j|j||}|ddt j||j} d} t|j |dz | |||} |j dkrd nd } |d z } td d d}| d| d |rW|d z }t|d5}tj|tj}dddn #1swxYwY|d}nd}t%t'|jt+| |}| ||d} | j}n&#t0$r}| t2jz}Yd}~nd}~wwxYw|d krt7jnt;|}t=||D]u\}}|d}| |z| |zkr'|d}|}|j | ||dd|zdd}| || d vdS)ayTrain pipeline Parameters ---------- protocol_name : `str` Name of pyannote.database protocol to use. subset : `str`, optional Use this subset for training. Defaults to 'development'. pretrained : Path, optional Use parameters in "pretrained" training directory to bootstrap the optimization process. In practice this will simply run a first trial with this set of parameters. n_iterations : `int`, optional Number of iterations. Defaults to 1. sampler : `str`, optional Choose sampler between RandomSampler and TPESampler pruner : `str`, optional Choose between MedianPruner or SuccessiveHalvingPruner. average_case : `bool`, optional Optimise for average case. Defaults to False (i.e. worst case). rprotocolr[Trexist_okr)defaulttrials.journal)db study_namer^r_r`minimizer rtrialr)unitpositionleavezFirst trial in progressr&moder'Nr,) warm_start show_progressloss)r,ruz Best trial: dg%)desc) r TRAIN_DIRr>rmkdirr get_protocolrIr r get_directionrset_descriptionupdater?r@rArBlistfilterrLgetattr tune_iter best_loss ValueErrornpinf itertoolscountrangezip dump_params)rOrZr[r\r]r^r_r`rrcrj optimizer directionr! progress_barpre_params_ymlrQ pre_paramsrsinputs iterationsrrUrr4statusru best_paramsrys r"trainzExperiment.trains5> N ! !#2& "     t444( )<     N++!%    5577:EEAA2 - 1DAAA $$%>???A  ',6Nn3/// C2!Yr$/BBB  C C C C C C C C C C C C C C C#H-JJJfT],EGHf,E,E,G,GHHII(( z)   +!+II + + +!BF*IIIIII +%1A%5%5 !!!5;N;NUJ// # #IAv&>D4)i"777$X.  **{+ 7# /6666D  ( (d ( 3 3 3    " " " " # #s*;!D((D,/D,F F:!F55F:ct|j|j||}d}t |j|dz |} |j}n'#t$r}tdYd}~dSd}~wwxYw|j }tdd|zd d tj |d } t| dS) a Print current best pipeline Parameters ---------- protocol_name : `str` Name of pyannote.database protocol used for training. subset : `str`, optional Subset used for training. Defaults to 'development'. rbrgrh)rirjz4Still waiting for at least one iteration to succeed.NzLoss = rvrwz&% with the following hyper-parameters:F)default_flow_style) rrzr>rr rrrprintrr@dump) rOrZr[rrjrrrUrcontents r"bestzExperiment.bestXs N ! !#2& "      Ny+;;     !+II    H I I I FFFFF  +  QiQQQQRRR)KEBBB gs A A9A44A9test output_dir use_filterctj||j} |j}n#t $r }d}Yd}~nd}~wwxYw|dd|r||d|d|jjz }n||d|d|jjz }t|d5} tt||} |rt|j | } d |d |d } t| | d D]l} || } |j| | | dd}|d}|Ot!| }||| |}m dddn #1swxYwY|jdz }|r||||d|d}t+|dS|r ||d|dz }n ||d|dz }t|d5} | t-|ddddS#1swxYwYdS)zApply current best pipeline Parameters ---------- protocol_name : `str` Name of pyannote.database protocol to process. subset : `str`, optional Subset to process. Defaults to 'test' rfNTrd.z _INCOMPLETE.wrqz Processing z ()file)iterableryrn annotation)uemlatestzWFor some (possibly good) reason, the output of this pipeline could not be evaluated on z_INCOMPLETE.evalz.eval)r r|rIr get_metricNotImplementedErrorr{ write_formatr?rrrrLrwriterDr parentexistsunlink symlink_torstr)rOrZrr[rrcmetricrU output_extrQfilesry current_fileoutput referencer_rmsg output_evals r"applyzExperiment.apply}s~$( )<     ^..00FF"   FFFFFF  555  "VVVVV9TVVW J  VVVV9TVVV *3 ' ' ' 7226224455E 5t}e44;;;&;;;D $e$V L L L 7 7  55$$R000),,\4@@ $!F>#L11F9f#666 7  7 7 7 7 7 7 7 7 7 7 7 7 7 7 70"X- ==??  MMOOO*%%% >G6CGGG  #JJJ F  H$-'R'R&'R'R'RRKK$-'G'G&'G'G'GGK +s # # "r HHS[[ ! ! ! " " " " " " " " " " " " " " " " " "s:7 A AA )CE;;E?E?#H??II)F)rYNr NNF)rY)rF)__name__ __module__ __qualname____doc__r=rz APPLY_DIR classmethodrrKr#r<rrintrrr __classcell__)rXs@r"rrs/Jz--subsetrrYz --foreverrlz --iterationsz --samplerz--prunerz --pretrainedT)strictz--average-casezr)r[r]r\r^r_r`rF)r[rrz --use-filterz z %Y%m%d-%H%M%S)rdate)r[r)rrsplitr load_databaserr expanduserresolverrrr#rr>rstrftimer) argumentsr-rZr[rr^r_r\r`r experimentrrrs r"mainrsw(;<<"F [ ! 8JJY~677JK(:&~.  Lj))4466>>d>KKJ !12 i(:;<<'2244<>>  #!%    6 >"Fi(:;<<'2244<F~. =122 ((**22$2?? ..y5.II   ' '#**;*D*D_*U*U (     :f      !  r$)rros.pathr@numpyrtypingrpathlibrrrrrpyannote.databaserr r pyannote.core.utils.helperr rr rrr1r$r"rsJ:XXt  ((((((&&&&&&++++++888888 E"E"E"E"E"E"E"E"P E E E E E r$