NLPCC 2019 Call for Participation (Shared Tasks)

The CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC) is the annual conference of CCF TCCI (Technical Committee of Chinese Information, China Computer Federation). The NLPCC conferences have been successfully held in Beijing (2012)，Chongqing (2013), Shenzhen (2014), Nanchang (2015), Kunming (2016), Dalian (2017) and Hohhot (2018). This year's NLPCC conference will be held in Dunhuang on October 9–14.

NLPCC 2019 will follow the NLPCC tradition of holding several shared tasks in natural language processing and Chinese computing. This year's shared tasks focus on both classic problems and newly emerging problems, including

◇ Task 1 - Cross-Domain Dependency Parsing

Organizer: Soochow University and Alibaba Inc.

Contact: Zhenghua Li (zhli13@suda.edu.cn) and Rui Wang (masi.wr@alibaba-inc.com)

◇ Task 2 - Open Domain Semantic Parsing

Organizer: Microsoft Research Asia.

Contact: Nan Duan (nanduan@microsoft.com)

◇ Task 3 - Open Domain conversation

Organizer: RSVP Technologies Inc..

Contact: Ying Shan (yshan@rsvp.ai)

The top 3 participating teams of each task will be certificated by NLPCC and CCF Technical Committee on Chinese Information Technology. If a task has multiple sub-tasks, then only the top 1 participating team of each sub-task will be certificated.

The detailed description of each task can be found in the task guidelines. Participants from both academia and industry are welcomed. Each group can participate in one or multiple tasks and members in each group can attend the NLPCC conference to present their techniques and results. The participants will be invited to submit papers to the main conference and the accepted papers will appear in the conference proceedings published by Springer LNCS.

1. Overview of the Shared Tasks

There are three shared tasks in this year’s NLPCC conference and the detailed description of each task can be found in the task guidelines released. Here we only give a brief overview of each task.

◇ Task 1 - Cross-Domain Dependency Parsing

With the surge of web data (or user generated content), cross-domain parsing has become the major challenge for applying syntactic analysis in realistic NLP systems. To meet the challenge of the lack of labeled data, we have manually annotated large-scale high-quality domain-aware datasets with a lot of effort (http://hlt.suda.edu.cn/index.php/SUCDT) in the past few years. We provide a source-domain labeled dataset (~20K sentences from balanced corpus), three target-domain labeled datasets (product blogs, product comments, and web fiction; ~25K in total), and large-scale unlabeled texts (size to be determined). We setup four sub-tasks with two cross-domain scenarios, i.e., semi-supervised (thousands of target-domain labeled data for training) and unsupervised (no target-domain labeled data for training), and two tracks, i.e., closed and open.

◇ Task 2 - Open Domain Semantic Parsing

The goal of this task is to predict the logical form (in lambda-calculus) of an input question based on a given knowledge graph. For example, for question “when was Bill Gates born?”, the predicted logical form should be . Each question in our dataset is annotated with entities, the question type and the corresponding logical form. We split this dataset into a train set, a development set and a test set. Both train and development sets will be provided to participating teams, while the test set will NOT. After participating teams submit their output files, we will evaluate their performances.

◇ Task 3 - Open Domain conversation

In NLPCC2019, we setup an open domain conversation task to evaluate human-computer conversations. All participating systems will be talking with human annotators, live user-in-the-loop. In the task, understanding natural language inputs (which can be questions or statements) is crucial, as well as providing smooth responses. The responses will be evaluated from five aspects. We will also provide human-annotated real data for researchers, to contribute to the community.

2. How to Participate

Please fill out the registration form and send it to the coordinators of the tasks by email.

If you have any question about the shared tasks, please do not hesitate to contact us (ws@pku.edu.cn and nanduan@microsoft.com)

3. Important dates

2019/03/15：announcement of shared tasks and call for participation;

2019/04/01：release of detailed task guidelines & training data release;

2019/05/15：test data release;

2019/05/20：participants’ results submission deadline;

2019/05/30：evaluation results release and call for system reports and conference papers;

2019/06/30：conference paper submission deadline (only for shared tasks);

2019/07/30：conference paper accept/reject notification;

2019/08/10：camera-ready paper submission deadline;