Time: 1.Prof. Shujian Huang and Wenhao Zhu (AM, 1st session)

Title: Research and Challenges of Multilingual Large Language Models

Abstract : Large language models have multilingual capabilities, which are reflected in their ability to translate between multiple languages and complete instructions in different languages. However, due to the limited training data, existing large language models often have significant limitations in their multilingual capabilities. On the one hand, many languages are not well supported; On the other hand, there are huge differences in ability across languages. This report introduces the current status of multilingual capabilities of large language models, and explores ways to make large language models serve more languages, including expanding their ability in new languages, narrowing the gap between English and other languages, and making large language model's multilingual capabilities interpretable. We anticipate that future research will enable large language models to serve different language users more fairly.

Speaker: Shujian Huang is currently an associate professor in Nanjing University. His research interests includes multilingual large language models, knowledge learning and reasoning in LLMs, etc. He is now the dupty director of the Techinical Committee on Machine Translation of CIPSC, and senior member of CCF. He was awarded the Excellent Young Scholar Research Project by Jiangsu Provincial Research Foundation in 2017, Outstanding Services by CIPSC in 2019, CCF-NLPCC Young Outstanding Scientist Award in 2020, CIPSC Hanwang Youth Innovation Award in 2022.

Speaker: Wenhao Zhu, is a final-year PhD student at the School of Computer Science, Nanjing University and a visiting PhD student at the Institute for Language, Cognition and Computation (ILCC), School of Informatics, University of Edinburgh. His research focuses on multilingual large language model and machine translation. He has presented his work at various international conferences, including ACL, NAACL, and EACL. He also serves as a reviewer for top-tier conferences and journals, such as ACL and TPAMI.

Prof. Ningyu Zhang (AM, 3rd session)

Title: Knowledge Mechanisms, Merging and Editing for Large Language Models

Abstract : Mastering knowledge has always been a core pursuit in the development of artificial intelligence systems. Large language models have demonstrated tremendous potential and have, to some extent, mastered and applied a wide range of knowledge. However, our understanding of how these models inherently acquire and store knowledge is still very limited, and we are unable to promptly correct errors and harmful knowledge within them. This talk will introduce the mechanisms of knowledge in large language models and cutting-edge methods for knowledge merging and knowledge editing.

Speaker: Ningyu Zhang, an associate professor at Zhejiang University and an Outstanding Young Scholar of the Qi Zhen Program at Zhejiang University, has published papers in high-impact international academic journals and conferences. Six of these papers have been selected as Paper Digest Most Influential Papers, and one has been chosen as a Featured Article in a Nature sub-journal. He has received the Second Prize for Scientific and Technological Progress from Zhejiang Province, KnowledgeNLP-ACL2024 Best Paper Award, IJCKG Best Paper Award, CCKS Best Paper Award. He has served as an Area Chair for ACL, EMNLP, ICLR, an Action Editor for ARR, and a Senior Program Committee Member for IJCAI. Additionally, he led the development of the knowledge editing tool for large language models, EasyEdit (1.7k stars).

Time: Prof. Juntao Li and Zecheng Tang (PM, 1st session)

Title: Long Context Modeling in LLMs: Advances and Challenges

Abstract : Long Context Modeling is the new frontier of LLMs, which is crucial in various application scenarios. Compared with existing short context models (SCMs), long context models (LCMs) present new challenges from different perspectives, including data strategy, modeling, and evaluation. This tutorial builds on recent advances of open-sourced LCMs to summarize critical resources (e.g., training data, evaluation benchmarks), introduce popular open-sourced models (e.g., Mamba, Mistral NeMo, LLaMA-3.1), and discuss opening challenges.

Speaker: Juntao Li is now an associate professor of Soochow University. He obtained a doctoral degree from Peking University in 2020. His research interests lie in text geneartion and language modeling. He has published 60+ papers on top-tier journals and conferences (e.g., TPAMI/NeurIPS/ICML/ICLR/ACL/EMNLP/COLM) and had given tutorials on IJCAI-19 and AAAI-20.

Speaker: Zecheng Tang is a Phd student of Soochow University, supervised by Prof. Min Zhang. He was awarded with the Star of Tomorrow Internship by MSRA. His research focuses on the long context modeling and generation. He has published papers at top-tier conferences and journals (with 600+ Google Scholar Citations), including ICML, ICLR, ACL, EMNLP, WSDM, and SCIS. He also serves as a reviewer for top-tier conferences, such as NeurIPS, ICLR, ACL. He is a core contributor of OpenBA-15B & OpenBA-V2 open-sourced models and Visual ChatGPT project (35000+ github stars).

Prof. Zhiliang Tian (PM, 2nd session)

Title: The hallucination and knowledge boundary of large language models

Abstract : Pre-trained large language models (LLMs) have achieved empirical results in many fields, but LLMs also often suffer from hallucination: making naive mistakes on some cases. The hallucination of LLMs is also a crucial issue that constrains the development of LLMs. After a detailed analysis of many examples of hallucination in LLMs, researchers found that the distribution of knowledge mastered by the LLMs has a significant impact on the generation of hallucination in LLMs. The degree of knowledge mastery by the LLMs can be delineated by knowledge boundaries, that is, the boundary between known knowledge and unknown knowledge is clarified. This report aims to introduce the manifestations, characteristics, and distribution of LLMs; hallucinations, analyze the causes of LLMs’ hallucinations, and introduce some detection or mitigation methods for LLMs’ hallucinations. Furthermore, from the perspective of the knowledge boundaries that LLM grasp, we study the relationship between the illusion of LLMs’ and their own mastery of knowledge, and introduce some representative works in this field. Finally, we summarize the characteristics ofLLMs’ hallucinations and analyze the future development directions for detecting or alleviating hallucinations.

Speaker: Tian Zhiliang, a core member of a national key laboratory in the School of Computer Science of the NUDT. He received his bachelor's degree from Harbin Institute of Technology and his Ph.D. degree from Hong Kong University of Science and Technology. He is committed to the research of large language models, text generation, and privacy protection. He has published more than 30 papers in artificial intelligence and natural language processing conferences such as NeurIPS, ACL, WWW, EMNLP, etc., including more than 20 first author or corresponding author papers, and has been authorized more than 20 national invention patents. He has been nominated for Microsoft Scholars, Baidu's highest award, and Baidu Scholarship Global Top 40. He has served as the area chair of ACL and EMNLP, a senior program member of IJCAI, and an executive member of the Youth Working Committee of CIPS. He was selected into the Young Elite Scientists Sponsorship Program by CAST.

Prof. Cuiyun Gao (AM, 2nd session)

Title: Exploration on LLM-Based Code Generation and Understanding

Abstract : LLMs are driving a paradigm shift in software development. As the complexity of software increases and market competition intensifies, LLM-based software development has become an inevitable trend. This tutorial will cover the recent explorations into LLMs for user requirement understanding, natural language programming, code completion, API recommendation, code review, and vulnerability detection. It also reflects on the role of LLMs in the software development.

Speaker: Cuiyun Gao is an Associate Professor at the Shool of Computer Science and Technology, Harbin Institute of Technology (Shenzhen). Her main research areas concentrate on NLP for Software Engineering towards realible and secure software. She has published more than 70 papers in prestigious journals and conferences including Transactions on Software Engineering (TSE), ACM Transactions on Software Engineering and Methodology (TOSEM), International Conference on Software Engineering (ICSE), and Foundations of Software Engineering (FSE), etc. She has received several awards which indicate her expertise in the area, including the Distinguished Paper Award at ICSE 2024 and ASE 2023, Best Paper Award of the Industry Challenge Track at ICSE 2024, and the Best Student Paper Award at IEEE ACAIT 2022.

Prof. Libo Qin (PM, 3rd session)

Title: Research and Challenges of Chain-of-Thought Reasoning

Abstract : Recently, Chain-of-thought reasoning in large language models gains increasing attention. It can be used to generate intermediate reasoning chains that lead to the final answer, not only improving the model's interpretability but also enhancing its reasoning capabilities in complex reasoning tasks. This report will provide a detailed overview of the development of chain-of-thought reasoning and introduce some cutting-edge directions, such as cross-lingual and cross-modal chain-of-thought development.

Speaker: Libo Qin is currently a professor at School of Computer Science and Engineering, Central South University. His research interests are natural language processing and large langauge models. He has published research papers at international NLP/AI conferences and journals, such as ACL, EMNLP, AAAI, and TASLP. His work has been selected as the Most Influential Paper by Paper Digest and won the Best Paper Award at the EMNLP2022 MMNLU Workshop. He has served as an Area Chair for EMNLP, NAACL, an Action Editor for ARR, and a Senior Program Committee Member for IJCAI.