Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
CoRR（2025）
关键词
Language Model,Large Language Models,Synchronization,Number Of Workers,Trainable Parameters,High-performance Computing,Communication Cost,Communication Overhead,GPU Memory,Oak Ridge,Model Parameters,Model Size,Model Weights,GB Memory,Space Complexity,Optimal State,Efficient Communication,Partitioning Scheme,Parallel Data,Forward Pass,High-performance Computing Systems,Backward Pass,Communication Volume,Oak Ridge National Laboratory,Large-scale Training,Hierarchical Strategy,NVIDIA GPU,Parallel Training,Topological Node,System Throughput
AI 理解论文
溯源树
样例
生成溯源树，研究论文发展脉络
Chat Paper
正在生成论文摘要