euroeval.dataset_configs

source package euroeval.dataset_configs

All dataset configurations used in EuroEval.

Functions

• get_all_dataset_configs — Get a mapping of all the dataset configurations.

source get_all_dataset_configs(custom_datasets_file: Path, dataset_ids: c.Sequence[str], api_key: str | None, cache_dir: Path, trust_remote_code: bool, run_with_cli: bool) → dict[str, DatasetConfig]

Get a mapping of all the dataset configurations.

Parameters

• custom_datasets_file : Path — A path to a Python file containing custom dataset configurations.

• dataset_ids : c.Sequence[str] — The IDs of the datasets to include in the mapping.

• api_key : str | None — The Hugging Face API key to use to check if the repositories have custom dataset configs.

• cache_dir : Path — The directory to store the cache in.

• trust_remote_code : bool — Whether to trust remote code from Hugging Face.

• run_with_cli : bool — Whether the code is being run with the CLI.

Returns

• dict[str, DatasetConfig] — A mapping between names of datasets and their configurations.