ZidZddlmZddlmZmZeGddZdeedeee effdZ d eee efdeefd Z d S) zEvaluation results utilities for the `.eval_results/*.yaml` format. See https://huggingface.co/docs/hub/eval-results for more details. Specifications are available at https://github.com/huggingface/hub-docs/blob/main/eval_results.yaml. ) dataclass)AnyOptionalc eZdZUdZeed<eed<eed<dZeeed<dZ eeed<dZ eeed<dZ eeed <dZ eeed <dZ eeed <dZeeed <dZeeed <ddZdS)EvalResultEntrya Evaluation result entry for the `.eval_results/*.yaml` format. Represents evaluation scores stored in model repos that automatically appear on the model page and the benchmark dataset's leaderboard. For the legacy `model-index` format in `README.md`, use [`EvalResult`] instead. See https://huggingface.co/docs/hub/eval-results for more details. Args: dataset_id (`str`): Benchmark dataset ID from the Hub. Example: "cais/hle", "Idavidrein/gpqa". task_id (`str`): Task identifier within the benchmark. Example: "gpqa_diamond". value (`Any`): The metric value. Example: 20.90. dataset_revision (`str`, *optional*): Git SHA of the benchmark dataset. verify_token (`str`, *optional*): A signature that can be used to prove that evaluation is provably auditable and reproducible. date (`str`, *optional*): When the evaluation was run (ISO-8601 datetime). Defaults to git commit time. source_url (`str`, *optional*): Link to the evaluation source (e.g., https://huggingface.co/spaces/SaylorTwift/smollm3-mmlu-pro). Required if `source_name`, `source_user`, or `source_org` is provided. source_name (`str`, *optional*): Display name for the source. Example: "Eval Logs". source_user (`str`, *optional*): HF user name for attribution. Example: "celinah". source_org (`str`, *optional*): HF org name for attribution. Example: "cais". notes (`str`, *optional*): Details about the evaluation setup. Example: "tools", "no-tools", "chain-of-thought". Example: ```python >>> from huggingface_hub import EvalResultEntry >>> # Minimal example with required fields only >>> result = EvalResultEntry( ... dataset_id="Idavidrein/gpqa", ... task_id="gpqa_diamond", ... value=0.412, ... ) >>> # Full example with all fields >>> result = EvalResultEntry( ... dataset_id="cais/hle", ... task_id="default", ... value=20.90, ... dataset_revision="5503434ddd753f426f4b38109466949a1217c2bb", ... verify_token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", ... date="2025-01-15T10:30:00Z", ... source_url="https://huggingface.co/datasets/cais/hle", ... source_name="CAIS HLE", ... source_org="cais", ... notes="no-tools", ... ) ``` dataset_idtask_idvalueNdataset_revision verify_tokendate source_url source_name source_user source_orgnotesreturnc`|j|j|j|jt ddSdS)NzaIf `source_name`, `source_user`, or `source_org` is provided, `source_url` must also be provided.)rrrr ValueError)selfs X/root/projects/butler/venv/lib/python3.11/site-packages/huggingface_hub/_eval_results.py __post_init__zEvalResultEntry.__post_init__UsI   (D,<,HDOLgo%s &%MhLg)rN)__name__ __module__ __qualname____doc__str__annotations__rr rr r rrrrrrrrrr s::xOOO LLL JJJ&*hsm***"&L(3-&&&D(3- $J $$$!%K#%%%!%K#%%% $J $$$E8C=rrentriesrcg}|D]}|j|jd}|j |j|d<||jd}|j |j|d<|j |j|d<|jAd|ji}|j |j|d<|j |j|d <|j |j |d <||d <|j |j |d <| ||S) aConvert a list of [`EvalResultEntry`] objects to a YAML-serializable list of dicts. This produces the format expected in `.eval_results/*.yaml` files. Args: entries (`list[EvalResultEntry]`): List of evaluation result entries to serialize. Returns: `list[dict[str, Any]]`: A list of dictionaries ready to be dumped to YAML. Example: ```python >>> from huggingface_hub import EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... EvalResultEntry(dataset_id="Idavidrein/gpqa", task_id="gpqa_diamond", value=0.412), ... ] >>> yaml_data = eval_result_entries_to_yaml(entries) >>> yaml_data[0] {'dataset': {'id': 'cais/hle', 'task_id': 'default'}, 'value': 20.9} ``` To upload eval results to the Hub: ```python >>> import yaml >>> from huggingface_hub import upload_file, EvalResultEntry, eval_result_entries_to_yaml >>> entries = [ ... EvalResultEntry(dataset_id="cais/hle", task_id="default", value=20.90), ... ] >>> yaml_content = yaml.dump(eval_result_entries_to_yaml(entries)) >>> upload_file( ... path_or_fileobj=yaml_content.encode(), ... path_in_repo=".eval_results/hle.yaml", ... repo_id="your-username/your-model", ... ) ``` )idr Nrevision)datasetr verifyTokenr urlnameuserorgsourcer) rr r r r r rrrrrappend)r!resultentryr%datar+s reval_result_entries_to_yamlr0^sRF).)9em"T"T  ! -"'"8GJ +2U[II   )"'"4D  : ! :DL   '&+U-=%>F ,!&!2v ,!&!2v+ % 0u #DN ; "!KDM d Mrr/cng}|D].}|d|}|di}|di}t|d|d|d|d|d|d |r|d nd |r|d nd |r|d nd |r|dnd |d }||0|S)aParse a list of dicts into [`EvalResultEntry`] objects. This parses the `.eval_results/*.yaml` format. For the legacy `model-index` format, use [`model_index_to_eval_results`] instead. Args: data (`list[dict[str, Any]]`): A list of dictionaries (e.g., parsed from YAML or API response). Returns: `list[EvalResultEntry]`: A list of evaluation result entry objects. Example: ```python >>> from huggingface_hub import parse_eval_result_entries >>> data = [ ... {"dataset": {"id": "cais/hle", "task_id": "default"}, "value": 20.90}, ... {"dataset": {"id": "Idavidrein/gpqa", "task_id": "gpqa_diamond"}, "value": 0.412}, ... ] >>> entries = parse_eval_result_entries(data) >>> entries[0].dataset_id 'cais/hle' >>> entries[0].value 20.9 ``` r/r%r+r#r r r$r&r r'Nr(r)r*r) rr r r r r rrrrr)getrr,)r/r!item entry_datar%r+r.s rparse_eval_result_entriesr5s88GXXfd++ ..B//"--t}W%I&$[[44# 66'',2 6***$.4> 6***$,2r:s "!!!!!  OOOOOOO OdCo)>C4SRUXCWCCCCL/Dc3h$8/T/=R//////r