gair-prox/DCLM-pro

Name: gair-prox/DCLM-pro
Creator: gair-prox
License: odc-by
Keywords: huggingface, task_categories:text-generation, language:en, license:odc-by, size_categories:100M<n<1B, format:parquet, modality:text, library:datasets, library:dask, text-generation

Text Generation · gair-prox· 16.4K

odc-by 2.9 TB

DCLM-pro is refined from DCLM using the ProX refining framework. It contains about 500B high quality tokens, ready for general language model pre-training.

Open in MLForge Sign up free Desktop app

# download instantly
mlforge datasets pull gair-prox/DCLM-pro

Dataset details

Task

Text Generation

Language

License

odc-by

Size

2.9 TB

Rows / images

365.6M

Creator

gair-prox

Downloads

16.4K

Source

huggingface_datasets

Updated

2025-02-15

About gair-prox/DCLM-pro

DCLM-pro is refined from DCLM using the ProX refining framework. It contains about 500B high quality tokens, ready for general language model pre-training.