jedZddlZddlZddlmZddlmZddlmZddl m Z ddl m Z ddlmZGd d Zdd ZGd deZdZddZdZdZdZdZdZdZedkr edSdS)z Tools for reading and writing dependency trees. The input is assumed to be in Malt-TAB format (https://stp.lingfil.uu.se/~nivre/research/MaltXML.html). N) defaultdict)chain)pformat) find_binary)open)TreeceZdZdZ d dZdZdZdZd Zd Z d Z d Z d Z dZ dZe d!dZdZdZdZ d"dZd#dZdZdZd$dZdZdZdZdZdZdZdS)%DependencyGraphzQ A container for the nodes and labelled edges of a dependency structure. NFROOTctd|_|jdddddd|_|r||||||dSdS)aDependency graph. We place a dummy `TOP` node with the index 0, since the root node is often assigned 0 as its head. This also means that the indexing of the nodes corresponds directly to the Malt-TAB format, which starts at 1. If zero-based is True, then Malt-TAB-like input with node numbers starting at 0 and the root node assigned -1 (as produced by, e.g., zpar). :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. c >dddddddttdd S)N) addresswordlemmactagtagfeatsheaddepsrel)rlist}/home/lighthouse/badge-speaker-benchmark/.benchmark-venv-community/lib/python3.11/site-packages/nltk/parse/dependencygraph.pyz*DependencyGraph.__init__..>s1#D))  rrTOP)rrrN)cell_extractor zero_basedcell_separatortop_relation_label)rnodesupdateroot_parse)selftree_strrrrr s r__init__zDependencyGraph.__init__%s0!       1 eEaHHIII   KK-%-#5        rc|j|=dS)zw Removes the node with the given address. References to this node in others will still exist. Nr!)r%rs rremove_by_addressz!DependencyGraph.remove_by_addressXs Jw   rc|jD]C}g}|dD]1}||vr||||2||d<DdS)zp Redirects arcs to any of the nodes in the originals list to the redirect node address. rN)r!valuesappend)r% originalsredirectnodenew_depsdeps r redirect_arcszDependencyGraph.redirect_arcs_s J%%'' $ $DHF| ) ))##OOH----OOC((((#DLL $ $rc|j|d}|j|d|g|j|d||dS)zw Adds an arc from the node specified by head_address to the node specified by the mod address. rrN)r! setdefaultr-)r% head_address mod_addressrelations radd_arczDependencyGraph.add_arcmsb :k*51 < (33HbAAA < (299+FFFFFrcH|jD]}|jD]k}|d|dkrW|ddkrK|d}|d|g|d||dldS)zr Fully connects all non-root nodes. All nodes are set to be dependents of the root node. rrrrN)r!r,r5r-)r%node1node2r8s r connect_graphzDependencyGraph.connect_graphws Z&&(( E EE**,, E E#uY'777E%LE>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> print(dg.to_dot()) digraph G{ edge [dir=forward] node [shape=plaintext] 0 [label="0 (None)"] 0 -> 2 [label="ROOT"] 1 [label="1 (John)"] 2 [label="2 (loves)"] 2 -> 1 [label=""] 2 -> 3 [label=""] 3 [label="3 (Mary)"] } z digraph G{ zedge [dir=forward] znode [shape=plaintext] c|dSNrr)vs rrz(DependencyGraph.to_dot..s a lr)keyz {} [label="{} ({})"]rrrNz {} -> {} [label="{}"]z {} -> {} z })sortedr!r,formatitems)r%sr0rrr2s rto_dotzDependencyGraph.to_dots 0  ## ''4:,,..4J4JKKK H HD )00YYV  A "&\//11 H H THHC6==d9osTWXXX]11$y/3GGG H H U rcH|}t|S)aShow SVG representation of the transducer (IPython magic). >>> from nltk.test.setup_fixt import check_binary >>> check_binary('dot') >>> dg = DependencyGraph( ... 'John N 2\n' ... 'loves V 0\n' ... 'Mary N 2' ... ) >>> dg._repr_svg_().split('\n')[0] '' )rMdot2img)r% dot_strings r _repr_svg_zDependencyGraph._repr_svg_s[[]] z"""rc*t|jSN)rr!r%s r__str__zDependencyGraph.__str__stz"""rc2dt|jdS)Nz)lenr!rTs r__repr__zDependencyGraph.__repr__s@DJ@@@@rct|5}fd|dDcdddS#1swxYwYdS)a :param filename: a name of a file in Malt-TAB format :param zero_based: nodes in the input file are numbered starting from 0 rather than 1 (as produced by, e.g., zpar) :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. :return: a list of DependencyGraphs c6g|]}t|S))rrr r ).0r&rr rs r z(DependencyGraph.load..sF  )#1'9 r N) _secure_openreadsplit)filenamerrr infiles ``` rloadzDependencyGraph.loads&( # # v!' 3 3F ; ;                  s5AAActj|j|d}|j|dt fd|DS)zl Returns the number of left children under the node specified by the given address. rrc3(K|] }|kdV dSNrr\cindexs r z0DependencyGraph.left_children..'44!e))1))))44rr from_iterabler!r,sumr% node_indexchildrenrks @r left_childrenzDependencyGraph.left_childrena &tz*'=f'E'L'L'N'NOO :&y14444h444444rctj|j|d}|j|dt fd|DS)zm Returns the number of right children under the node specified by the given address. rrc3(K|] }|kdV dSrgrris rrlz1DependencyGraph.right_children..rmrrnrqs @rright_childrenzDependencyGraph.right_childrenrurc||ds(|j|d|dSdSrF)rCr!r")r%r0s radd_nodezDependencyGraph.add_nodesK$$T)_55 5 JtI ' . .t 4 4 4 4 4 5 5rc Dd}d}d}d} |||| d} t|trd|dD}d|D} d | D} d } t| d D]D\} }||}| t |} n| t |ksJ|? | | }n5#t $r(}t d | |d }~wwxYw ||| \} }}}}}}}n*#tt f$r||\}}}}}}}YnwxYw|dkrt|}|r|d z }|j |  | |||||||d| dkr|dkr|}|j |d| | F|j dd|r:|j dd|d}|j ||_ ||_d Stjdd S)aParse a sentence. :param extractor: a function that given a tuple of cells returns a 7-tuple, where the values are ``word, lemma, ctag, tag, feats, head, rel``. :param str cell_separator: the cell separator. If not provided, cells are split by whitespace. :param str top_relation_label: the label by which the top relation is identified, for examlple, `ROOT`, `null` or `TOP`. c"|\}}}|||||d|dfSNr)cellsrkrrrs rextract_3_cellsz/DependencyGraph._parse..extract_3_cellss$#OD#t$c3D"< .extract_4_cells s'#( D#tS$c3D#= =rcl|\}}}}}}} t|}n#t$rYnwxYw|||||d||fSr}int ValueError) rrk line_indexrrr_rrs rextract_7_cellsz/DependencyGraph._parse..extract_7_cells$sa9> 6JeS!T3 J    $sCT3> >s  ))c r|\ }}}}}}}} } } t|}n#t$rYnwxYw|||||||| fSrSr) rrkrrrrrrrrrs rextract_10_cellsz0DependencyGraph._parse..extract_10_cells-sgIN FJeT3tS!Q J    $tS%sB Bs  ,,) c3K|]}|VdSrSr)r\lines rrlz)DependencyGraph._parse..>s"::td::::::r c3>K|]}|VdSrS)rstripr\ls rrlz)DependencyGraph._parse..@s*,,,,,,,,rc3K|]}||V dSrSrrs rrlz)DependencyGraph._parse..As'''qQ'''''''rNrh)startTNumber of tab-delimited fields ({}) not supported by CoNLL(10) or Malt-Tab(4) formatr)rrrrrrrrrrrzBThe graph doesn't contain a node that depends on the root element.) isinstancestrra enumeraterWKeyErrorrrJ TypeErrorrr!r"r-r#r warningswarn)r%input_rrrr rrrr extractorslines cell_numberrkrrerrrrrrr root_addresss rr$zDependencyGraph._parses , = = = > > > ? ? ? C C C    fc " " ;::v||D'9'9:::F,,V,,,''E''' $U!4441 81 8KE4JJ~..E"!%jj "c%jj0000%%/ %%;P;P8eT3tSSS  Q s{{t99D   Ju  $ $$ " "      q  tqyy( Jt V $S ) 0 0 7 7 7 7 :a= !3 4 :a=01CDQGL <0DI&8D # # # MW     s*8C C3 #C..C37D  $D43D4Tc*|d}|r|dkr|S|S)Nr,r)r%r0filterws r_wordzDependencyGraph._words& L  Cxxrc|}|d}ttj|d}|rt |fd|DS|S)zTurn dependency graphs into NLTK trees. :param int i: index of a node :return: either a word (if the indexed node is a leaf) or a ``Tree``. rrc:g|]}|Sr_treer\r2r%s rr]z)DependencyGraph._tree..s#???3tzz#???r)rArIrror,r)r%ir0rrs` rrzDependencyGraph._trees{ ""1%%F|e)$v,*=*=*?*?@@AA  ????$???@@ @Krcj}|d}ttj|d}t |fd|DS)z Starting with the ``root`` node, build a dependency tree using the NLTK ``Tree`` constructor. Dependency labels are omitted. rrc:g|]}|Srrrs rr]z(DependencyGraph.tree..s#;;;s4::c??;;;r)r#rIrror,r)r%r0rrs` rtreezDependencyGraph.treesb yF|e)$v,*=*=*?*?@@AAD;;;;d;;;<<>> dg = DependencyGraph(treebank_data) >>> dg.contains_cycle() False >>> cyclic_dg = DependencyGraph() >>> top = {'word': None, 'deps': [1], 'rel': 'TOP', 'address': 0} >>> child1 = {'word': None, 'deps': [2], 'rel': 'NTOP', 'address': 1} >>> child2 = {'word': None, 'deps': [4], 'rel': 'NTOP', 'address': 2} >>> child3 = {'word': None, 'deps': [1], 'rel': 'NTOP', 'address': 3} >>> child4 = {'word': None, 'deps': [3], 'rel': 'NTOP', 'address': 4} >>> cyclic_dg.nodes = { ... 0: top, ... 1: child1, ... 2: child2, ... 3: child3, ... 4: child4, ... } >>> cyclic_dg.root = top >>> cyclic_dg.contains_cycle() [1, 2, 4, 3] )rrhrNF)rrrr!iterrkr-pop) r%WHITEGRAYBLACKcolorrpathstackr2s rcontains_cyclezDependencyGraph.contains_cyclesdP%tUC  $*%%  EU|u$$E%L7D$*U+F3445E  9  C$*,, SzT)) $DJJsOO$5$5666666SzU**%)c  C((( T$*S/&*A%B%BCCC + ).E$((**%IIKKK "urc |dD]}||kr |dgcS|dD]^}||||}t|dkr |d|d|cS_gS)Nrrr)get_cycle_pathrArWinsert)r% curr_nodegoal_node_indexr2rs rrzDependencyGraph.get_cycle_pathsV$ . .Co%%!),----&V$  C&&t':':3'?'?QQD4yy1}} Ay3444  rc|dkrdn4|dkrdn+|dkrdn"td|dfd t|jDS) z The dependency graph in CoNLL format. :param style: the style to use for the format (3, 4, 10 columns) :type style: int :rtype: str rz{word} {tag} {head} rz{word} {tag} {head} {rel} rz9{i} {word} {lemma} {ctag} {tag} {feats} {head} {rel} _ _ rr~c3TK|]"\}}|ddkjdd|i|V#dS)rrrNr)rJ)r\rr0templates rrlz+DependencyGraph.to_conll..#sY  4E{e## HO ( (a (4 ( (####  r)rrJjoinrIr!rK)r%stylers @rto_conllzDependencyGraph.to_conll s A::0HH aZZ7HH b[[U H228&--  ww    !$*"2"2"4"455      rcTddl}ttdtj}fd|D}i_|D]}j|dj|<|}|||||S)zJConvert the data in a ``nodelist`` into a networkx labeled directed graph.rNrhcg|]B}||||fCSr)rr)r\nr%s rr]z,DependencyGraph.nx_graph...sS   /0488A;;  TYYq\\ *   rr) networkxrrangerWr! nx_labels MultiDiGraphadd_nodes_fromadd_edges_from)r%r nx_nodelist nx_edgelistrgs` rnx_graphzDependencyGraph.nx_graph)s5C OO4455     4?    6 6A $ 1 f 5DN1    ! ! # # %%% %%%r)NNFNr )FNr )NFNr )TrS)__name__ __module__ __qualname____doc__r'r*r3r9r=rArCrMrQrUrX staticmethodrdrtrxrzr$rrrrrrrrrrrrrr r s ! 1111f    $ $ $GGG E E E(((***+++Z### ###AAALR\:555555555! xxxxt    = = = . . . . BBBH      :rr svgcp td}n"#t$r}td|d}~wwxYw |dvrtj|d|zgd|d}n*tj|d|zgt |d  }|jS#t$r#td |wxYw) a Create image representation fom dot_string, using the 'dot' program from the Graphviz package. Use the 't' argument to specify the image file format, for ex. 'jpeg', 'eps', 'json', 'png' or 'webp' (Running 'dot -T:' lists all available formats). Note that the "capture_output" option of subprocess.run() is only available with text formats (like svg), but not with binary image formats (like png). dotz0Cannot find the dot binary from Graphviz packageN)rdot_jsonjsonrz-T%sT)capture_outputinputtextutf8)encoding)rzACannot create image representation by running dot from string: {})r LookupError Exception subprocessrunbytesstdoutrJ)rPt dot_binaryrprocs rrOrO<s S !'' SSSJKKQRRS 2 2 2>VaZ(#  DD>VaZ(J888D{     vj!!    s 1,1AB-B5ceZdZdZdS)DependencyGraphErrorzDependency graph exception.N)rrrrrrrrres%%%%rrcvttttdSrS) malt_demo conll_democonll_file_democycle_finding_demorrrdemoris2 KKKLLLrFctd}|}||rddl}ddlm}|}|||d}| ||d| |||j | g| g|d |dSdS) zw A demonstration of the result of reading a dependency version of the first sentence of the Penn Treebank. Pierre NNP 2 NMOD Vinken NNP 8 SUB , , 2 P 61 CD 5 NMOD years NNS 6 AMOD old JJ 2 NMOD , , 2 P will MD 0 ROOT join VB 8 VC the DT 11 NMOD board NN 9 OBJ as IN 9 VMOD a DT 15 NMOD nonexecutive JJ 15 NMOD director NN 12 PMOD Nov. NNP 9 VMOD 29 CD 16 NMOD . . 9 VMOD rN)pylabrh)dim2) node_sizeztree.png)r rpprintr matplotlibrrinfo spring_layoutdraw_networkx_nodesdraw_networkx_labelsrxticksytickssavefigshow)nxdgrrrrposs rrrps    B* 7799DKKMMM $$$$$$ KKMM $$QA$..$$Qr$:::%%abl;;; R R j!!! rctt}|}|t |t |ddS)zg A demonstration of how to read a string representation of a CoNLL format dependency tree. rN)r conll_data1rr printr)rrs rrrsT  % %B 7799DKKMMM "III "++a..rctddtdD}|D]9}|}td|:dS)NzMass conll_read demo...c0g|]}|t|Srr[)r\entrys rr]z#conll_file_demo..s% U U Uu Uoe$$ U U Urr^r)r conll_data2rarr )graphsgraphrs rrrsr #$$$ U U+2C2CF2K2K U U UFzz|| d  rctt}t|t}|ddgddd|ddgddd|ddgddd|ddgddd|ddgdddt|dS) Nrhrr)rrrrrNTOPrr)r treebank_datarrrz)r cyclic_dgs rrrs  ' 'B "    !!I qc%ANNOOO qc&QOOPPP qc&QOOPPP qc&QOOPPP qc&QOOPPP ) " " $ $%%%%%rra/ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ a1 Cathy Cathy N N eigen|ev|neut 2 su _ _ 2 zag zie V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 hen hen Pron Pron per|3|mv|datofacc 2 obj1 _ _ 4 wild wild Adj Adj attr|stell|onverv 5 mod _ _ 5 zwaaien zwaai N N soort|mv|neut 2 vc _ _ 6 . . Punc Punc punt 5 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 had heb V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 met met Prep Prep voor 8 mod _ _ 4 haar haar Pron Pron bez|3|ev|neut|attr 5 det _ _ 5 moeder moeder N N soort|ev|neut 3 obj1 _ _ 6 kunnen kan V V hulp|ott|1of2of3|mv 2 vc _ _ 7 gaan ga V V hulp|inf 6 vc _ _ 8 winkelen winkel V V intrans|inf 11 cnj _ _ 9 , , Punc Punc komma 8 punct _ _ 10 zwemmen zwem V V intrans|inf 11 cnj _ _ 11 of of Conj Conj neven 7 vc _ _ 12 terrassen terras N N soort|mv|neut 11 cnj _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Dat dat Pron Pron aanw|neut|attr 2 det _ _ 2 werkwoord werkwoord N N soort|ev|neut 6 obj1 _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 ze ze Pron Pron per|3|evofmv|nom 6 su _ _ 5 zelf zelf Pron Pron aanw|neut|attr|wzelf 3 predm _ _ 6 uitgevonden vind V V trans|verldw|onverv 3 vc _ _ 7 . . Punc Punc punt 6 punct _ _ 1 Het het Pron Pron onbep|neut|zelfst 2 su _ _ 2 hoorde hoor V V trans|ovt|1of2of3|ev 0 ROOT _ _ 3 bij bij Prep Prep voor 2 ld _ _ 4 de de Art Art bep|zijdofmv|neut 6 det _ _ 5 warme warm Adj Adj attr|stell|vervneut 6 mod _ _ 6 zomerdag zomerdag N N soort|ev|neut 3 obj1 _ _ 7 die die Pron Pron betr|neut|zelfst 6 mod _ _ 8 ze ze Pron Pron per|3|evofmv|nom 12 su _ _ 9 ginds ginds Adv Adv gew|aanw 12 mod _ _ 10 achter achter Adv Adv gew|geenfunc|stell|onverv 12 svp _ _ 11 had heb V V hulp|ovt|1of2of3|ev 7 body _ _ 12 gelaten laat V V trans|verldw|onverv 11 vc _ _ 13 . . Punc Punc punt 12 punct _ _ 1 Ze ze Pron Pron per|3|evofmv|nom 2 su _ _ 2 hadden heb V V trans|ovt|1of2of3|mv 0 ROOT _ _ 3 languit languit Adv Adv gew|geenfunc|stell|onverv 11 mod _ _ 4 naast naast Prep Prep voor 11 mod _ _ 5 elkaar elkaar Pron Pron rec|neut 4 obj1 _ _ 6 op op Prep Prep voor 11 ld _ _ 7 de de Art Art bep|zijdofmv|neut 8 det _ _ 8 strandstoelen strandstoel N N soort|mv|neut 6 obj1 _ _ 9 kunnen kan V V hulp|inf 2 vc _ _ 10 gaan ga V V hulp|inf 9 vc _ _ 11 liggen lig V V intrans|inf 10 vc _ _ 12 . . Punc Punc punt 11 punct _ _ 1 Zij zij Pron Pron per|3|evofmv|nom 2 su _ _ 2 zou zal V V hulp|ovt|1of2of3|ev 7 cnj _ _ 3 mams mams N N soort|ev|neut 4 det _ _ 4 rug rug N N soort|ev|neut 5 obj1 _ _ 5 ingewreven wrijf V V trans|verldw|onverv 6 vc _ _ 6 hebben heb V V hulp|inf 2 vc _ _ 7 en en Conj Conj neven 0 ROOT _ _ 8 mam mam V V trans|ovt|1of2of3|ev 7 cnj _ _ 9 de de Art Art bep|zijdofmv|neut 10 det _ _ 10 hare hare Pron Pron bez|3|ev|neut|attr 8 obj1 _ _ 11 . . Punc Punc punt 10 punct _ _ 1 Of of Conj Conj onder|metfin 0 ROOT _ _ 2 ze ze Pron Pron per|3|evofmv|nom 3 su _ _ 3 had heb V V hulp|ovt|1of2of3|ev 0 ROOT _ _ 4 gewoon gewoon Adj Adj adv|stell|onverv 10 mod _ _ 5 met met Prep Prep voor 10 mod _ _ 6 haar haar Pron Pron bez|3|ev|neut|attr 7 det _ _ 7 vriendinnen vriendin N N soort|mv|neut 5 obj1 _ _ 8 rond rond Adv Adv deelv 10 svp _ _ 9 kunnen kan V V hulp|inf 3 vc _ _ 10 slenteren slenter V V intrans|inf 9 vc _ _ 11 in in Prep Prep voor 10 mod _ _ 12 de de Art Art bep|zijdofmv|neut 13 det _ _ 13 buurt buurt N N soort|ev|neut 11 obj1 _ _ 14 van van Prep Prep voor 13 mod _ _ 15 Trafalgar_Square Trafalgar_Square MWU N_N eigen|ev|neut_eigen|ev|neut 14 obj1 _ _ 16 . . Punc Punc punt 15 punct _ _ __main__)r)F)rrr collectionsr itertoolsrr rnltk.internalsr nltk.pathsecrr_ nltk.treerr rOrrrrrrrr!rrrrrrr)s ######&&&&&&------YYYYYYYYx& & & & R&&&&&9&&&****Z    & & & ( T l zDFFFFFr