j6dZddlZddlmZddlmZgdZ ddlmZddl m Z ddl m Z d Zn#e$rd ZGd d eZYnwxYwd ZdZ dZ dS)aO XML parsing helpers that refuse entity declarations. ``xml.etree.ElementTree`` honours ```` declarations in a document's internal DTD subset, so a small file can expand to an arbitrarily large string in memory -- the "billion laughs" / entity-expansion DoS (CWE-776). A 330-byte document expands to a megabyte with five levels of nesting, and each further level multiplies by ten. libexpat 2.6.0 (2024) added an input-amplification cap, but it only engages above an activation threshold (~8 MiB of output by default) and belongs to whatever libexpat the running interpreter links -- not to NLTK. Builds against an older system libexpat have no cap at all. These helpers make the guarantee NLTK's own. :func:`parse` and :func:`fromstring` use ``defusedxml`` when it is installed and fall back to an equivalent standard-library implementation when it is not, so NLTK is protected either way and ``defusedxml`` stays an optional dependency. External entities are *not* the concern here -- ``ElementTree`` does not resolve them (it reports ``undefined entity``) -- but the fallback rejects them too, for parity with ``defusedxml``'s ``forbid_external`` default. What is rejected ---------------- Any document that *declares* an entity (general or parameter) or references an external one. A DOCTYPE that merely names an external DTD (````) is *allowed*, because real corpora use them and ElementTree never fetches the external subset anyway. The five predefined XML entities (``&`` ``<`` ``>`` ``"`` ``'``) and numeric character references need no declaration and keep working. Both back ends agree on this policy. Why the fallback re-parses with expat instead of screening text --------------------------------------------------------------- An earlier fallback screened the DOCTYPE internal subset with string walking. That is a parser *differential* waiting to happen: a ``z{. Entity expansion can exhaust memory (CWE-776); if this document is trusted, parse it with xml.etree.ElementTree directly.)namesuper__init__)selfr args __class__s n/home/lighthouse/badge-speaker-benchmark/.benchmark-venv-community/lib/python3.11/site-packages/nltk/xmlsec.pyrzEntitiesForbidden.__init__KsKDI GG  T5H[     )r )__name__ __module__ __qualname____doc__r __classcell__)rs@rrrDsG            rrctj}d}d}||_||_ ||ddS#tj$rYdSwxYw)aRaise :class:`EntitiesForbidden` if ``data`` declares/refers an entity. Runs a standalone expat parse whose entity handlers raise on the first declaration or external reference. ``data`` is fed to expat in its own type (``str`` or ``bytes``) so the pre-scan and the real ElementTree parse decode identically. A malformed document raises ``ExpatError`` here; that is left for ElementTree to report so the error matches a direct parse, and it is not exploitable because the declaration would never be reached. c t|)Nr )r rests r_forbid_declarationz-_reject_entities.._forbid_declarationas%%%rc td)Nexternalr )rs r_forbid_externalz*_reject_entities.._forbid_externalds +++rTN)r ParserCreateEntityDeclHandlerExternalEntityRefHandlerParse ExpatError)dataparserrr s r_reject_entitiesr(Us  ! !F&&&,,, 3F&6F#  T4           sAAActtrt|St|tj|S)z<``ElementTree.fromstring`` that refuses entity declarations.)r_defused_fromstringr(rr)texts rrrqs7)"4(((T  !$ ' ''rctrt|St|dr|}nrBs --^ !!!!!! I I I,,,,,,HHHHHH>>>>>>OOOJ(   8(((%%%%%s-AA