◇ Lecture 2: Recent Progress on Deep Learning for NLP

◇  Zhengdong Lu: Senior Researcher, Founder and CTO of DeeplyCourious.Ai

◇ ABSTRACT: Dr. Lu will give a talk on the recent progress of deep learning for NLP (DL4NLP) as a particular direction of Deep Learning. More specifically he will talk briefly about the basic concepts of DL4NLP, and then he will elaborate on the recent progress on differentiable data structure and learning paradigms.

◇ Short Bio: Zhengdong Lu is a founder and CTO of DeeplyCourious.Ai. Before that he was a senior researcher at Noah’s Ark Lab, Huawei. Before joining Huawei, Dr. Lu worked as an associate researcher in MSRA and postdoctoral researcher at University of Texas at Austin. He got his bachelor’s degree from Xi’an Institute of Post and Telecommunication, and Ph.D degree from Oregon Health & Science University. His research interest includes deep learning，natural language processing, reasoning and general machine learning. Dr. Lu has authored over 40 papers on NIPS, ICML, ACL, and JMLR etc.

◇ Lecture 3: Efficient Training and Deployment of Large Scale Deep Learning Systems for NLP

◇  Jacob Devlin, Principal NLP Scientist, Microsoft Research

◇ ABSTRACT: Large-scale, complex neural networks such as attentional sequence-to-sequence models for machine translation are often slow to train and very memory intensive, even on powerful GPUs. Additionally, it is often difficult to deploy the trained model running on the CPU (or GPU) with acceptable latency for a real time service.
In this tutorial, I will detail several techniques for drastically improving model training time and memory usage on a single GPU (or a small number of GPUs). These techniques can improve the training time for various recurrent neural network architectures architecture by 5x or more. They also allow much wider and deeper networks to be trained on a single GPU.
I will also describe a number of tricks improving the speed of the networks on the CPU at inference time, with a specific focus on attentional sequence-to-sequence models.

◇ Short Bio: Jacob Devlin is a Principal NLP Scientist in the Machine Translation Group at Microsoft Research in Redmond, Washington. His primary research focus is on developing fast, scalable deep learning systems for machine translation and other applications. Prior to joining Microsoft, Mr. Devlin worked as a Research Scientist at BBN Technologies in Cambridge, Massachussets. His work on machine translation received the ACL best paper award in 2014 and the NAACL best paper award in 2012. He received his B.A./M.S. in Computer Science from the University of Maryland, College Park in 2009.

◇ Lecture 5: NLP Technologies in Question Answerers and Voice Assistants

◇  Dekang Lin (co-founder and co-CEO of naturali.io.), Jidian Jizhi

◇ ABSTRACT: Question answerers and voice assistants are two of the most important NLP applications. The former aims at providing precise, direct answers answers to users questions. The latter goes a step further by not only figuring out the semantic interpretation of a request, but also execute a sequence of actions to fulfil the request on the user's behalf. Both applications require their NLP components to be precise, robust, as well as efficient. I will survey and discuss the NLP technologies used in these two applicaitons.

◇ Short Bio: Dekang Lin is co-founder and co-CEO of naturali.io, a Beijing-based startup for voice assistant. He was a Senior Staff Research Scientist at Google from 2004 to 2016, where he led a team of researchers and engineers to build a question-answerer in Google search.
Before joining Google, Dekang Lin was a full professor of Computer Science at University of Alberta. He authored 90 papers with over 12000 citations. He was elected as Fellow of Association for Computational Linguistics (ACL) in 2012 and served as the program co-chair and general chair for ACL-2002 and ACL-2011 respectively.

◇ Lecture 6: Deep Learning for Answering Questions

◇  Lei Li, Toutiao

◇ ABSTRACT: Human language is still elusive for machines to comprehend. In recent years, there are much effort in pushing the performance of machines in complex tasks such as dialog and question answering. In this talk, we will review recent progress of deep recurrent neural networks in answering factoid questions regarding large knowledge bases. We will give a deep dive to our CFO system. It builds upon techniques essential to success of retrieving candidate answers from a Knowledge Base given a question: including word meaning in vector space, entity mentions in an utterance, parsing user intents in queries, and linking entities and relations in a large-scale knowledge base. It can retrieve answers for questions such as “who is the creator of Harry Potter” from tens of millions of facts.

◇ Short Bio: Lei Li is a research scientist at Toutiao and director of Toutiao Lab. Lei received his B.S. in Computer Science and Engineering from Shanghai Jiao Tong University (ACM class) and Ph.D. in Computer Science from Carnegie Mellon University, respectively. His dissertation work on fast algorithms for mining co-evolving time series was awarded ACM KDD best dissertation (runner up). Before Toutiao, he worked at Baidu's Institute of Deep Learning in Silicon Valley as a Principal Research Scientist (Young Leading Scholar). Previously, he worked briefly at Microsoft Research (Asia and Redmond), Google (Mountain View), and IBM (TJ Watson Reserch Center). Before joining Baidu, he was working in EECS department of UC Berkeley as a Post-Doctoral Researcher. His research interest lies in the intersection of deep learning, statistical inference, natural language understanding, and time series analysis.

◇ Lecture 7: Fundamentals and Challenges in Mining Social Media

◇  Huan Liu, Computer Science and Engineering at Arizona State University

◇ ABSTRACT: This ADL lecture consists of three parts: (1) Fundamentals in social media mining, (2) some recent algorithmic developments, and (3) data and evaluation challenges. We start with the basics of social media data and mining and show how it is different from traditional data mining. We will provide a brief review based on the Cambridge University Press textbook on Social Media Mining. Participants can access the text at the speaker’s website. In the second part of this lecture, we introduce some interesting and ingenious ideas and algorithms and demonstrate that research opportunities are where challenges are. In the last part of this lecture, we discuss some current challenges we are facing: where we can find data to suit our needs, how we can know whether data is sufficient if we crawl data ourselves, how we can separate wheat from chaff and sieve misinformation in search of knowledge, and how we can evaluate our findings where ground truth is not readily available. Questions are most welcome.