Journal of High Energy Physics(2023)
Université Clermont Auvergne | Czech Academy of Sciences | Goethe University Frankfurt | European Organization for Nuclear Research | Polytechnic University of Turin | INFN Sezione di Bologna | Homi Bhabha National Institute | Aligarh Muslim University | Korea Institute of Science & Technology Information | University of Pavol Jozef Šafárik | GSI Helmholtz Centre for Heavy Ion Research | Bose Institute | INFN Sezione di Torino | Central China Normal University | Universidad Nacional Autónoma de México | AGH University of Krakow | University of Houston | University of Bergen | INFN Sezione di Cagliari | Horia Hulubei National Institute for R and D in Physics and Nuclear Engineering | University of Münster | Heidelberg University | INFN Sezione di Padova | Lawrence Berkeley National Laboratory | IMT Atlantique | Université Grenoble Alpes | Yale University | INFN Sezione di Catania | Sungkyunkwan University | Gangneung–Wonju National University | University of Science and Technology of China | Indian Institute of Technology Indore | Institut de Recherche sur les Lois Fondamentales de l'Univers | University of Jammu | INFN Sezione di Bari | Technical University of Munich | National and Kapodistrian University of Athens | Wigner Research Centre for Physics | Daresbury Laboratory | Hospital Universitário da Universidade de São Paulo | University of Liverpool | Lund University | Nantes Université | Indian Institute of Technology Bombay | Pontifical Catholic University of Peru | Centre National de la Recherche Scientifique | Sejong University | University of Oslo | Institute of Space Science | Variable Energy Cyclotron Centre | Czech Technical University in Prague | Institute of Nuclear Physics | Laboratoire de Physique Subatomique et des Technologies Associées | The University of Texas at Austin | Frankfurt Institute for Advanced Studies | University of Pavia | Oak Ridge National Laboratory | Inha University | INFN Sezione di Pavia | Université Paris-Saclay | INFN Sezione di Trieste | Stefan Meyer Institute for Subatomic Physics | National Research Foundation | Autonomous University of Sinaloa | Universitatea Națională de Știință și Tehnologie Politehnica București | Institut National de Physique Nucléaire et de Physique des Particules | Benemérita Universidad Autónoma de Puebla | Université Claude Bernard Lyon 1 | Universidade Estadual de Campinas (UNICAMP) | National Institute for Subatomic Physics | University of Copenhagen | University of Tsukuba | University of Birmingham | Laboratoire de Physique des 2 Infinis Irène Joliot-Curie | Commissariat à l'Énergie Atomique et aux Énergies Alternatives | University of Salerno | Warsaw University of Technology | University of California | Centro de Aplicaciones Tecnológicas y Desarrollo Nuclear | University of Cape Town | Panjab University | Technical University of Košice | Comenius University Bratislava | University of Zagreb | National Laboratory of Frascati | Western Norway University of Applied Sciences | Laboratoire de Physique Subatomique et de Cosmologie | Saga University | Chicago State University | University of Kansas | Universidade Federal do Rio Grande do Sul | University of Tennessee System | Nagasaki Institute of Applied Science | University of Split | Utrecht University | University of Bonn | Center for Research and Advanced Studies of the National Polytechnic Institute | Université de Strasbourg | Yonsei University | Creighton University | University of Jyväskylä | The Ohio State University | COMSATS University Islamabad | China Institute of Atomic Energy | Suranaree University of Technology | Institute of Experimental Physics | KTO Karatay University | University of Trieste | Pusan National University | Jeonbuk National University | Hiroshima University | Sofia University "St. Kliment Ohridski" | National Centre for Nuclear Research | University of South-Eastern Norway | Fudan University | National Nuclear Energy Agency of Indonesia | National Institute of Science Education and Research | Helsinki Institute of Physics | The University of Tokyo | University of the Witwatersrand | Chungbuk National University | University of Tennessee at Knoxville | University of Brescia | Polytechnic University of Bari | Universidade Federal do ABC | National Nuclear Research Center | Institute of Physics | Wayne State University | University of Tübingen | Nara Women's University | Saha Institute of Nuclear Physics | Bogolyubov Institute for Theoretical Physics | National Academy of Sciences of Ukraine | United Nations University Institute for Sustainability and Peace
- Pretraining has recently greatly promoted the development of natural language processing (NLP)
- We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
- We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
- The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
- Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance

被引用791 | 浏览
被引用55 | 浏览