Distributed Sign Momentum with Local Steps for Training Transformers

Shaoqing Yu(Intelligent Health (United Kingdom)), Soummya Kar(Carnegie Mellon University), Ding Zhou(Orion Corporation (United Kingdom)), An Xu(Chinese Academy of Sciences), Zhi Feng Zhang, Xin Liu(Xuzhou Medical College), Cong Xie
arXiv (Cornell University)
November 26, 2024
Cited by 0


Related Papers