Pre-training Text Representations as Meta Learning

Shangwen Lv(Chinese Academy of Sciences), Songlin Hu(Chinese Academy of Sciences), Nan Duan(Microsoft Research Asia (China)), Daxin Jiang(Jiangsu University), Daya Guo(Sun Yat-sen University), Linjun Shou(Microsoft Research Asia (China)), Ming Zhou(Langdon Hospital), Ming Gong(Microsoft Research Asia (China)), Yuechen Wang(Peking University), Duyu Tang(Microsoft Research Asia (China)), Ryan Ma, Guihong Cao(Microsoft (Finland)), Fuqing Zhu
arXiv (Cornell University)
April 12, 2020
10.48550/arxiv.2004.05568
Cited by 7

Related Papers

CodeBERT: A Pre-Trained Model for Programming and Natural Languages
|Unknown|2020|2.5k
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
|Proceedings of the AAAI Conference on Artificial Intelligence|2020|744
CLIP4Clip: An empirical study of CLIP for end to end video clip retrieval and captioning
|Neurocomputing|2022|676
DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
|Nature|2025|610
UniXcoder: Unified Cross-Modal Pre-training for Code Representation
|Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)|2022|554