RoBERTa

RoBERTa: A Robustly Optimized BERT Pretraining Approach. RoBERTa iterates on BERT’s pretraining procedure, including training the model longer, with bigger batches over more data; removing the next sentence prediction objective; training on longer sequences; and dynamically changing the masking pattern applied to the training data. See the associated paper for more details.


References in zbMATH (referenced in 29 articles )

Showing results 1 to 20 of 29.
Sorted by year (citations)

1 2 next

  1. Aghahadi, Zeinab; Talebpour, Alireza: Avicenna: a challenge dataset for natural language generation toward commonsense syllogistic reasoning (2022)
  2. Feiliang Ren, Yongkang Liu, Bochao Li, Shilei Liu, Bingchao Wang, Jiaqi Wang, Chunchao Liu, Qi Ma: An Understanding-Oriented Robust Machine Reading Comprehension Model (2022) arXiv
  3. Koto, Fajri; Baldwin, Timothy; Lau, Jey Han: FFCI: a framework for interpretable automatic evaluation of summarization (2022)
  4. Kriebel, Johannes; Stitz, Lennart: Credit default prediction from user-generated text in peer-to-peer lending using deep learning (2022)
  5. Li, Zuchao; Zhou, Junru; Zhao, Hai; Zhang, Zhisong; Li, Haonan; Ju, Yuqi: Neural character-level syntactic parsing for Chinese (2022)
  6. Loureiro, Daniel; Mário Jorge, Alípio; Camacho-Collados, Jose: LMMS reloaded: transformer-based sense embeddings for disambiguation and beyond (2022)
  7. Md Tanvirul Alam, Dipkamal Bhusal, Youngja Park, Nidhi Rastogi: CyNER: A Python Library for Cybersecurity Named Entity Recognition (2022) arXiv
  8. Quamar, Abdul; Efthymiou, Vasilis; Lei, Chuan; Özcan, Fatma: Natural language interfaces to data (2022)
  9. Wu, Xueqing; Xia, Yingce; Zhu, Jinhua; Wu, Lijun; Xie, Shufang; Qin, Tao: A study of BERT for context-aware neural machine translation (2022)
  10. Xie, Shufang; Xia, Yingce; Wu, Lijun; Huang, Yiqing; Fan, Yang; Qin, Tao: End-to-end entity-aware neural machine translation (2022)
  11. Zeng, Zhiyuan; Xiong, Deyi: Unsupervised and few-shot parsing from pretrained language models (2022)
  12. Zhang, Hongming; Liu, Xin; Pan, Haojie; Ke, Haowen; Ou, Jiefu; Fang, Tianqing; Song, Yangqiu: ASER: towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities (2022)
  13. Zhang, Wei; Xue, Ziyun; Li, Zhong; Yin, Huichao: DCE-dforest: a deep forest model for the prediction of anticancer drug combination effects (2022)
  14. Juan Manuel Pérez, Juan Carlos Giudici, Franco Luque: pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks (2021) arXiv
  15. Kiritchenko, Svetlana; Nejadgholi, Isar; Fraser, Kathleen C.: Confronting abusive language online: a survey from the ethical and human rights perspective (2021)
  16. Nelson Tavares de Sousa, Wilhelm Hasselbring: JavaBERT: Training a transformer-based model for the Java programming language (2021) arXiv
  17. Sai Muralidhar Jayanthi, Kavya Nerella, Khyathi Raghavi Chandu, Alan W Black: CodemixedNLP: An Extensible and Open NLP Toolkit for Code-Mixing (2021) arXiv
  18. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, Shujie Liu: CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation (2021) arXiv
  19. Škrlj, Blaž; Martinc, Matej; Lavrač, Nada; Pollak, Senja: autoBOT: evolving neuro-symbolic representations for explainable low resource text classification (2021)
  20. Tao Gui, Xiao Wang, Qi Zhang, Qin Liu, Yicheng Zou, Xin Zhou, Rui Zheng, Chong Zhang, Qinzhuo Wu, Jiacheng Ye, Zexiong Pang, Yongxin Zhang, Zhengyan Li, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xinwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Bolin Zhu, Shan Qin, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang: TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021) arXiv

1 2 next