MMLForge Search models, datasets, tasks… Start free
Home › Models › Fill Mask › moussaKam/mbarthez
M

moussaKam/mbarthez

Fill Mask·moussaKam· 107.7K· 8
transformers apache-2.0 Fill Mask arxiv:2010.12321license:apache-2.0deploy:azureregion:us

A french sequence to sequence pretrained model based on BART. BARThez is pretrained by learning to reconstruct a corrupted input sentence. A corpus of 66GB of french raw text is used to carry out the pretraining. Unlike already existing BERT-based French language models such as CamemBERT and FlauBERT, BARThez is particularly well-suited for generative tasks (such as abstractive summarization), s

Open in MLForge Sign up free Desktop app Source ↗
# pull & run locally
pip install mlforge-sdk && mlforge pull moussaKam/mbarthez

Model details

Task
Fill Mask
Provider
moussaKam
Framework
transformers
Size
5.1 GB
License
apache-2.0
Downloads
107.7K
Likes
8
Paper
arXiv:2010.12321
Updated
2024-02-20

About moussaKam/mbarthez

A french sequence to sequence pretrained model based on BART. BARThez is pretrained by learning to reconstruct a corrupted input sentence. A corpus of 66GB of french raw text is used to carry out the pretraining. Unlike already existing BERT-based French language models such as CamemBERT and FlauBERT, BARThez is particularly well-suited for generative tasks (such as abstractive summarization), since not only its encoder but also its decoder is pretrained.

Related Fill Mask

B google-bert/bert-base-uncased Fill Mask ·110.1M params 75.4M 2.7K 🤗 HF X FacebookAI/xlm-roberta-base Fill Mask ·278.9M params 21.1M 853 🤗 HF R FacebookAI/roberta-base Fill Mask ·124.7M params 11.9M 617 🤗 HF R FacebookAI/roberta-large Fill Mask ·355.4M params 11.5M 301 🤗 HF M answerdotai/ModernBERT-base Fill Mask ·149.7M params 9.4M 1.1K 🤗 HF D distilbert/distilbert-base-uncased Fill Mask ·67.0M params 8.9M 901 🤗 HF

Browse all Fill Mask models →