◆  Industrial speech, by Fuzheng Zhang

Topic: 「KwaiYii」 Foundation Models: Decoding Information and Embraceing Intelligence

Date and Time: Oct.14

Short Bio: Dr. Zhang Fuzheng, the head of Kuaishou's Natural Language Processing Center and Audio Center, is also in charge of the 「Kwai-Yii」Foundation Models Project. He has long been involved in the construction of AI technologies such as Natural Language Processing, multimodal processing, and knowledge graphs, as well as their specific applications in business scenarios. He has published over 60 papers in top-tier conferences and journals in related fields like KDD, WWW, ACL, EMNLP, and has more than 10,000 Google Scholar citations. He has served as reviewers for relevant academic activities for an extended period. Dr. Zhang has been named on Stanford University's "The World's Top 2% Scientists List", the "AI 2000 Most Influential Scholars in Artificial Intelligence for the Year 2023," and the "2019 Benchmark Figures in Knowledge Graphs for Artificial Intelligence in China".

Abstract: With the release of ChatGPT and GPT-4, large-scale foundation models are demonstrating accelerated iterative cycles and substantial improvements in capabilities even at high performance levels. These models are exhibiting performance that surpasses average human levels and, in some domains, even reaches the pinnacle of human capability, signaling the potential for immense social and commercial impact. 「KwaiYii」is a series of large-scale foundation models, including Large Language Models (LLMs), domain-specific large models, and multi-modal large models, developed from scratch by the Kuaishou AI team. 「KwaiYii」excels in foundational technology and have achieved state-of-the-art results in the majority of authoritative Chinese and English benchmarks such as MMLU, C-Eval, CMMLU, and HumanEval, all while being comparable in size to other available models. Featuring excellent language comprehension and generation capabilities, 「KwaiYii」supports a wide array of tasks, including content creation, information consultation, mathematical logic, code writing, and multi-turn dialogue. Human evaluations indicate that 「KwaiYii」has reached industry-leading standards, particularly in Chinese language contexts.

◆  Industrial speech, by Tianhuang Su

Topic: New Breeno: AI Assistant Powered by Large Language Models

Date and Time: Oct.14

Short Bio: Su Tianhuang is the Head of Natural Language Processing and Pretraining Team in OPPO Breeno Intelligent Center. Breeno is the AI assistant on OPPO smartphones and IoT devices. Tianhuang graduated from Sun Yat-sen University, and joined OPPO in 2020 after working as an algorithm engineer in Baidu. His main research interests include natural language proccessing, dialog conversation and search engine, and has published papers in top conferences and journals such as KDD, etc. Currently, he is committed to leading Breeno's NLP and Pretraining team to realize a smarter dialog system with Large Language Models.

Abstract: In this talk, a general introduction to Breeno, the AI assistant on OPPO smartphones and IoT devices, will be given first. Then Tianhuang will share the two stages of how Breeno adopts large language models in its dialog system. The first stage refers to OBERT series based on bert architecture for natural language understanding, and the secord stage AndesGPT series are based on decoder-only architecture. In addition, he will share serveral useful skills (e.g. universal represetation) for project engineering on dialog system.

◆  Industrial workshop speech, by Linlin Li

Topic: AI Foundation Models at Huawei Noah’s Ark Lab

Date and Time: Oct.14

Short Bio: Linlin Li is a senior research scientist at Huawei Noah’s Ark Lab. Her major research interest is in the field of Natural Language Processing. Linlin Li received her Ph.D degree in Computer Science from Saarland University, Germany, with a dissertation titled Computational Modelling of Lexical Ambiguity. She joint Microsoft soon after graduation and worked in Europe for several years before moving back to China and joining Alibaba in Hangzhou. She joint Huawei in 2019 and has been working on NLP related projects in Huawei ever since. She has published over 30 papers in top NLP conferences and journals such as ACL, EMNLP, NAACL and Computational Linguistics.

Abstract: The latest progress on LLMs has drawn enormous research interest in large language modelling among the artificial intelligence community. In this talk, we introduce our latest progress on LLM-based foundation models including Pangu-alpha, Pangu-sigma, and other multimodal based approaches. Meanwhile, we also introduce how we integrate knowledge via information retrieval and how we extend the capability of the raw LLMs by integrating various tools. The talk is started by an introduction of the foundation model Pangu-alpha, followed by Pangu-sigma which is a MoE version of the LLM that aims at efficient compute. Then, we introduce our work on how to leverage web/domain search components and various tools/plugins into our model to solve complex tasks. In the last part, we give a brief introduction of multimodal approaches which process not only text-based info but also image- and audio-based content.

◆  Industrial workshop speech, by Junyang Lin

Topic: Qwen: Open Foundation, Human-Aligned and Specialist Models

Date and Time: Oct.14

Short Bio: Junyang Lin is a staff engineer in Alibaba Group. He graduated from Peking University. His research interests are on natural language processing and multimodal representation learning, with a focus on large-scale pretraining. He has published articles on NeurIPS, ICML, ACL, etc. Previously, he developed the extremely large-scale pretrained model M6, unified multimodal multitask model OFA, cross-modal representation model Chinese CLIP, etc. Recently, he is leading the development of the large language model, Qwen, and working on pretraining, alignment, multimodal integration and AI agent.

Abstract: LLMs has remarkable potentials to use external tools and understand multimodal data like humans. Recently we have publicly released Qwen-7B and Qwen-14B, as well as their assistant models, Qwen-7b-Chat and Qwen-14B-Chat. These models demonstrate strong performance and significantly outperform the baselines on a series of benchmark. We reinforce the models on tool use and thus they have the potential to become powerful AI agents for downstream applications. Additionally, based on the Qwen models, we continue pretraining on code and math data, and produce the specialist models Code-Qwen and Math-Qwen, which pave a way towards training domain-specific experts. In this talk, I will give a brief introduction to our models and the techniques that make a difference in building a strong LLM.