Task
Fill Mask
C
yikuan8/Clinical-Longformer
Fill Mask·yikuan8· 510.5K· 69
transformers
arxiv:2201.11838deploy:azureregion:us
Clinical-Longformer is a clinical knowledge enriched version of Longformer that was further pre-trained using MIMIC-III clinical notes. It allows up to 4,096 tokens as the model input. Clinical-Longformer consistently out-performs ClinicalBERT across 10 baseline dataset for at least 2 percent. Those downstream experiments broadly cover named entity recognition (NER), question answering (QA), natur
# pull & run locally
pip install mlforge-sdk && mlforge pull yikuan8/Clinical-Longformer
pip install mlforge-sdk && mlforge pull yikuan8/Clinical-Longformer
Model details
Provider
yikuan8
Framework
transformers
Size
1.1 GB
Downloads
510.5K
Likes
69
Paper
arXiv:2201.11838
Updated
2023-01-24
About yikuan8/Clinical-Longformer
Clinical-Longformer is a clinical knowledge enriched version of Longformer that was further pre-trained using MIMIC-III clinical notes. It allows up to 4,096 tokens as the model input. Clinical-Longformer consistently out-performs ClinicalBERT across 10 baseline dataset for at least 2 percent. Those downstream experiments broadly cover named entity recognition (NER), question answering (QA), natural language inference (NLI) and text classification tasks. For more details, please refer to our paper. We also provide a sister model at Clinical-BigBIrd