anisoleai/fineweb-tokenized

Name: anisoleai/fineweb-tokenized
Creator: anisoleai
License: odc-by
Keywords: huggingface, task_categories:text-generation, language:en, license:odc-by, size_categories:n>1T, modality:tabular, modality:text, arxiv:2406.17557, region:us, text-generation

Text Generation · anisoleai· 131.5K

odc-by 7.5 TB task_categories:text-generationlanguage:enlicense:odc-bysize_categories:n>1Tmodality:tabular

4 trillion tokens of the pre-tokenized data the 🌐 web has to offer

Open in MLForge Sign up free Desktop app

# download instantly
mlforge datasets pull anisoleai/fineweb-tokenized

Dataset details

Task

Text Generation

Language

License

odc-by

Size

7.5 TB

Rows / images

4132.0B

Creator

anisoleai

Downloads

131.5K

Source

huggingface_datasets

Updated

2026-05-29

About anisoleai/fineweb-tokenized

4 trillion tokens of the pre-tokenized data the 🌐 web has to offer