tasksource/mmlu
MMLU (hendrycks_test on huggingface) without auxiliary train. It is much lighter (7MB vs 162MB) and faster than the original implementation, in which auxiliary train is loaded (+ duplicated!) by default for all the configs in the original version, making it quite heavy. We use this version in tasksource. Reference to original dataset: Measuring Massive Multitask Language Understanding - https://github.com/hendrycks/test @article{hendryckstest2021, title={Measuring Massive Multitask Language… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/mmlu.
mlforge datasets pull tasksource/mmlu
Dataset details
About tasksource/mmlu
MMLU (hendryckstest on huggingface) without auxiliary train. It is much lighter (7MB vs 162MB) and faster than the original implementation, in which auxiliary train is loaded (+ duplicated!) by default for all the configs in the original version, making it quite heavy.