j1dZddlmZddlZddlZddlZddlmZddlm Z m Z dZ Gddej Z Gd d ej Zdd d d"dZdZehdZGddej ZehdZdddd#d!ZdS)$aZ Helpers for safer and/or more explicit pickle usage in NLTK. - RestrictedUnpickler: blocks unpickling of *any* globals (classes/functions). Intended for loading NLTK data packages where we control the serialization. - WarningUnpickler: emits a security warning before unpickling (does not make unpickling safe). - AllowlistUnpickler: only reconstructs an explicit, audited allowlist of globals. Use it for loading objects whose set of legitimate classes is known (e.g. a trained model) so that arbitrary-code gadgets such as ``os.system`` can never be reconstructed from an untrusted file. ) annotationsN)Iterable)AnyBinaryIOzSecurity warning: loading pickles can execute arbitrary code. Only load pickle files from trusted sources and never from untrusted or unauthenticated locations.ceZdZdZd dZdS) RestrictedUnpicklerzW Unpickler that prevents any class or function from being used during loading. modulestrnamereturnrc8tjd|d|d)Nglobal '.z' is forbidden)pickleUnpicklingErrorselfr r s q/home/lighthouse/badge-speaker-benchmark/.benchmark-venv-community/lib/python3.11/site-packages/nltk/picklesec.py find_classzRestrictedUnpickler.find_class*s'$%M%M%M%M%M%MNNNNr r r r r r)__name__ __module__ __qualname____doc__rrrrr%s8OOOOOOrrc6eZdZdZddd fd Zdfd ZxZS)WarningUnpicklerz6Unpickler that emits PICKLE_WARNING once per instance.Ncontextfilerr str | Nonekwargsrc Xtj|fi|||_d|_dS)NF)super__init___context_warned)rr!r r# __class__s rr&zWarningUnpickler.__init__2s4(((((  rr c|jsC|jtntd|jd}tj|t dd|_t S)Nz ()) stacklevelT)r(r'PICKLE_WARNINGwarningswarnRuntimeWarningr%load)rmsgr)s rr2zWarningUnpickler.load7sk| =(&::$-:::  M#~! < < < <DLww||~~r)r!rr r"r#r)r r)rrrrr&r2 __classcell__r)s@rrr/sn@@@D          rrF)r restrictedr!rr r"r6boolr rc|r!t|St||S)z Convenience wrapper similar to pickle.load(file). - If restricted=True, uses RestrictedUnpickler (no warning by default). - If restricted=False, uses WarningUnpickler and emits a warning. r)rr2r)r!r r6s r pickle_loadr9CsE0"4((--/// D' 2 2 2 7 7 9 99r)osntposix subprocesssyssocketshutilsignalptypopen2commandsctypes importlibrunpybuiltins webbrowserpdbbdbcodecodeopmultiprocessingasyncio threadingrz numpy.f2pyznumpy.ctypeslibznumpy.distutilsz numpy.testingznltk.tokenize.reppznltk.internals> numpyr2rRsaverRsavezrRmemmaprRloadtxtrRsavetxtrRfromfilerR fromregexrR vectorizerR DataSourcerR genfromtxtrRsavez_compressedpandas read_picklec`eZdZdZddddfd ZeddZddZdfd ZddZ xZ S)AllowlistUnpickleraF Unpickler that only reconstructs an explicit, audited allowlist of globals. Two complementary allowlists are supported: - ``allowed_globals``: a set of exact ``(module, qualname)`` pairs. Use this for individual safe globals from an otherwise-dangerous module, e.g. ``("builtins", "int")``. - ``allowed_modules``: a set of module names; a global is permitted when its module equals an allowed name or is a submodule of one (i.e. ``module`` equals ``name`` or starts with ``name + "."``). Three hard guards apply *before* the allowlists and cannot be opted out of: 1. A dotted ``name`` is rejected. For pickle protocol >= 4, ``pickle.Unpickler.find_class`` resolves a dotted ``name`` through :func:`getattr` chaining, so an attacker who is allowed the module ``sklearn`` could otherwise request ``sklearn.os.system`` and reach a command-execution sink the module allowlist never intended (GHSA-4489). Legitimate model globals are always a single qualname. 2. Globals from a denied module (:data:`_DENIED_MODULE_PREFIXES`) are rejected even if a parent namespace is allowlisted, so an in-namespace gadget cannot ride a broad ``allowed_modules`` entry (GHSA-x99w). 3. ``builtins`` is denied wholesale; individual safe builtins must be named explicitly via ``allowed_globals``. Anything not covered by either allowlist raises ``UnpicklingError`` before the global is resolved, so dangerous callables such as ``os.system`` or ``builtins.eval`` can never be reconstructed from an untrusted pickle. This is stricter than :func:`pickle_load` (which only warns and then executes) but, unlike :class:`RestrictedUnpickler` (which blocks *all* globals), it still allows the known-good classes a saved object legitimately needs. rallowed_globalsallowed_modulesr!rroIterable[tuple[str, str]]rp Iterable[str]r#rc tj|fi|t|tr|f}t ||_t ||_dS)N)r%r& isinstancer set_allowed_globalstuple_allowed_modules)rr!rorpr#r)s rr&zAllowlistUnpickler.__init__sd ((((( os + + 1.0O #O 4 4 %o 6 6rr r r r7cDtfdtDS)Nc3TK|]"}|kp|dzV#dSrN startswith).0denyr s r z4AllowlistUnpickler._module_denied..R   dN ;f//s ;;      r)any_DENIED_MODULE_PREFIXES)r s`r_module_deniedz!AllowlistUnpickler._module_denieds;    /      rcDtfd|jDS)Nc3TK|]"}|kp|dzV#dSr{r|)r~r r s rrz5AllowlistUnpickler._module_allowed..rr)rrx)rr s `r_module_allowedz"AllowlistUnpickler._module_alloweds>    -      rr c t||}t|tjrt jd|d|dt|dd}t|d|}||ftvr8|r6| |r!t jd|d|d|d|d ||ftvr!t jd|d|d|d|d |S) NrrzL' resolves to a module object, which is not a reconstructable class/functionrrz ' re-exports z from a denied modulez' (zK) is a denied callable and cannot be reconstructed from an untrusted pickle) r%rrttypes ModuleTyperrgetattr_SAFE_DENIED_GLOBALSr_DENIED_GLOBALS)rr r obj true_module true_qualnamer)s r_resolvezAllowlistUnpickler._resolvesvgg  .. c5+ , , (7677D777 c<66 ^T::   '/C C C t22;?? ,6>>D>>>  ??4  T]]4%8%8 (Q6QQDQQQ  D>_ , ,(9699D999  D>T2 2 2##F++ TN*>>>,@v@@@@@==.. .   v & & (9699D999     ' ' /==.. .$ Fv F F F F F   r)r!rrorqrprrr#r)r r r r7r) rrrrr& staticmethodrrrrr4r5s@rrmrms  L68)+ 7 7 7 7 7 7 7 7   \     @+ + + + + + + + rrm> rHintrHrurHr rHr7rHdictrHlistrHbytesrHfloatrHrwrHcomplexrH frozensetrrnrorqrprrcJt|||S)a~ Load a pickle while only permitting an explicit allowlist of globals. See :class:`AllowlistUnpickler` for the meaning of ``allowed_globals`` and ``allowed_modules``. Prefer this over the warn-only :func:`pickle_load` when the set of legitimate classes in the file is known (e.g. a trained model), because :func:`pickle_load` still executes arbitrary code. rn)rmr2)r!rorps rallowlisted_pickle_loadr3s+  o    dff r)r!rr r"r6r7r r)r!rrorqrprrr r)r __future__rrrr/collections.abcrtypingrrr. Unpicklerrrr9rrrrmrrrrrrs  #""""" $$$$$$ $OOOOO&*OOOv'*.2e : : : : : :*N)&I I I I I )I I I `!y   (24%'         r