NLPCC 2023 will follow the NLPCC tradition of holding several shared tasks in natural language processing and Chinese computing. This year’s shared tasks focus on both classical problems and newly emerging problems, including Chinese Grammatical Error Correction, Multi-perspective Scientific Machine Reading Comprehension, Math Word Problem Solving, Conversational Aspect-based Sentiment Quadruple Analysis, Chinese Medical Instructional Video Question Answering, Chinese Few-shot and Zero-shot Entity Linking, Chinese Essay Discourse Coherence Evaluation, Chinese Spelling Check, User Feedback Prediction and Response Generation, and Multimodal Dialogue Understanding and Response Prediction.

Participants from both academia and industry are welcomed. Each group can participate in one or multiple tasks. The participants will be invited to submit papers to the main conference and the accepted papers will appear in the conference proceedings published by Springer LNCS.

The top 3 participating teams of each task will be certificated by NLPCC and CCF-NLP. If a task has multiple sub-tasks or tracks, only the top 1 participating team of each sub-task/track will be certificated.

There are ten shared tasks in this year’s NLPCC conference and the detailed description of each task can be found in the task guidelines released later. Here we provide a brief overview of each task.

◇ Task 1 - Chinese Grammatical Error Correction

Chinese Grammatical Error Correction (CGEC) aims to automatically correct grammatical errors that violate language rules and convert the noisy input texts to cleaned output texts. This task focuses on correcting grammatical errors made by Chinese native speakers.

Organizer: Tsinghua Shenzhen International Graduate School, Tsinghua University, and Peng Cheng Laboratory

Contact: Haitao ZHENG (masr21@mails.tsinghua.edu.cn)

◇ Task 2 - Multi-perspective Scientific Machine Reading Comprehension

Scientific machine reading comprehension (SMRC) aims to understand scientific texts through interactions with humans by given questions. In this task, questions about scientific texts include perspectives from beginners, students and experts. It requires different levels of understanding of scientific texts.

Organizer: School of Computer Science and Technology, Beijing Institute of Technology

Contact: Heqi ZHENG (3220201024@bit.edu.cn)

◇ Task 3 - Math Word Problem Solving

In the math word problem solving task, the objective is to develop a model that can automatically read and comprehend a math word problem and then produce a numerical solution. To achieve this goal, models must effectively understand both the natural language text and underlying mathematical concepts that are necessary for solving the problem.

Organizer: Tencent AI Lab

Contact: Jipeng ZHANG (jzhanggr@connect.ust.hk)

◇ Task 4 - Conversational Aspect-based Sentiment Quadruple Analysis

This task aims to detect all sentiment quadruples (target, aspect, opinion, sentiment) in a given dialogue, where the first three items are text spans in the utterances, and the last item is a category label. The dialogue may involve multiple threads and speakers, and the quadruples may span multiple utterances, meaning the quadruple elements may come from different utterances. The dataset used for the shared task includes both Chinese and English languages, which are manually annotated.

Organizer: School of Cyber Science and Engineering, Wuhan University, and National University of Singapore

Contact: Fei LI (lifei_csnlp@whu.edu.cn)

◇ Task 5 - Chinese Medical Instructional Video Question Answering

Recently, the availability of online videos has revolutionized the way to access information or gain knowledge. Many people find instructional videos to be an effective and efficient way to teach or learn how to accomplish a particular task with a series of step-by-step procedures. To this end, a new task temporal answer grounding in the Chinese Medical instructional video (TAGV) is proposed to find the video frame span corresponding to an input question. The ultimate goal for this shared task is to develop a system that can provide video temporal answers for the first-aid, medical emergency, or medical education. The datasets of the shared task are collected from high-quality Chinese medical instructional channels on the YouTube website, which are manually annotated by medical experts.

This shared task includes three tracks:

● Track 1: Temporal Answer Grounding in Single Video (TAGSV): Given a medical or health-related question and a single untrimmed Chinese medical instructional video, this track aims to locate the temporal answer (start and end time points) within the video.

● Track 2: Video Corpus Retrieval (VCR): Given a medical or health-related question and a large collection of untrimmed Chinese medical instructional videos, this track aims to find the most relevant video corresponding to the given question in the video corpus.

● Track 3: Temporal Answer Grounding in Video Corpus (TAGVC): Given a medical or health-related question and a large collection of untrimmed Chinese medical instructional videos, this track aims at finding the matching video answer span with the corresponding question from the video corpus.

Organizer: College of Electrical and Information Engineering, Hunan University, National Laboratory of Pattern Recognition Institute of Automation, CAS, and Xiangya Hospital of Central South University

Contact: Bin LI (libincn@hnu.edu.cn)

◇ Task 6 - Chinese Few-shot and Zero-shot Entity Linking

Entity Linking (EL) is the task of grounding a textual mention in context to a corresponding entity in a knowledge base (KB). However, current EL systems demonstrate a popularity bias, significantly underperforming on tail and emerging entities. To this end, this task aims at testing the generalization ability of Chinese EL systems to less popular and newly emerging entities. The dataset for this task is a human-calibrated and multi-domain Chinese EL benchmark with Wikidata as KB, consisting of few-shot and zero-shot test sets.

Organizer: Harbin Institute of Technology (Shenzhen), and Tencent

Contact: Zhenran XU (xuzhenran@stu.hit.edu.cn)

◇ Task 7 - Chinese Essay Discourse Coherence Evaluation

In the scoring of the Chinese National College Entrance Examination (NCEE) and the Senior High School Entrance Examination, essay assessment is the most time-consuming and controversial task. While existing research has focused on language factors such as characters, words, and sentences, it has not explored the relationship between discourse coherence and text quality. The logical structure and coherence within an essay are essential for evaluation, but the lack of large-scale, high-quality discourse coherence evaluation data resources has hindered the development of AI essay grading. To address this issue, the CubeNLP laboratory of East China Normal University and Microsoft have constructed a Chinese essay coherence evaluation dataset called LEssay, which provides high-quality data resources and is significant for the development of automatic essay evaluation.

This shared task includes four tracks:

● Track 1: Coherence Evaluation (CE). Given a middle school student essay, annotators will assess its coherence on a three-level scale of excellent, moderate, and poor. A score of 2 indicates excellent coherence, 1 indicates moderate coherence, and 0 indicates incoherence.

● Track 2: Text Topic Extraction (TTE). Given a middle school student essay, annotators need to identify the topic sentence for each paragraph and one main topic sentence for the whole essay.

● Track 3: Paragraph Logical Relation Recognition (PLRR). Given two paragraphs sorted in order from a composition, the annotator needs to determine the logical relationship between the two paragraphs based on the given definitions and examples of logical relationships.

● Track 4. Sentence Logical Relation Recognition (SLRR). Given two sentences from an essay that are ordered sequentially, the annotator needs to determine what type of logical relation exists between them based on given definitions and examples.

Organizer: East China Normal University, and Microsoft Research Asia

Contact: Hongyi WU (why_hongyiwu@163.com)

◇ Task 8 - Chinese Spelling Check

Chinese Spelling Check (CSC) aims to detect and correct error tokens in Chinese contexts, which has a wide range of applications. In today's highly developed media environment, various types of news emerge constantly, but the quality of related content varies greatly. For instance, some news contains spelling errors and improper word usage. Therefore, this task aims to correct spelling errors such as homophonic errors, visually similar errors, and other types of errors in sentences from Chinese news articles.

Organizer: Peking University, and Beijing Founder Electronics Co., Ltd.

Contact: Xunjian YIN (xjyin@pku.edu.cn)

◇ Task 9 - User Feedback Prediction and Response Generation

Online conversation systems usually have a user feedback mechanism, such as like and dislike buttons. When a user is satisfied with the response, they can click the like button, and vice versa for the dislike button. The feedback signal represents the user's vote on the quality of the response and also represents their preference. It is a worthwhile direction to study and invest in how to use this signal to improve the quality of the conversation system.

● Track 1: Prediction of likes and dislikes: Given a (query, reply) pair, predict the probabilities of likes, dislikes, and no feedback.

● Track 2: Conversation generation based on likes and dislikes: Incorporate like and dislike data into conversation generation to improve response quality and obtain high likes.

Organizer: Renmin University of China, and XiaoMi AI Lab

Contact: Shuang DONG (dongshuang1@xiaomi.com)

◇ Task 10 - Learn to Watch TV: Multimodal Dialogue Understanding and Response Prediction

The multimodal dialogue understanding and response prediction task can be divided into two phases: multimodal context understanding and response prediction. Specifically, the former includes dialogue session identification (i.e., determining whether the dialogue content has changed) and dialogue scene identification (i.e., determining whether the video context has changed). The ultimate goal is to predict a response that is coherent with the dialogue topic and relevant to the video scene.

● Track 1: Dialogue scene identification: predict the boundaries of different dialogue scenes given a set of continuous dialogue utterances and a related video.

● Track 2: Dialogue session identification: predict the boundaries of different dialogue sessions given a set of continuous dialogue utterances and a related video (which is identical to Track 1).

● Track 3: Dialogue response retrieval: retrieve a response based on scene and session predictions, while coherently catching up with the conversation.

● Track 4. Dialogue response generation: generate a response based on scene and session predictions, while coherently catching up with the conversation.

Organizer: Peking University

Contact: Yueqian Wang (wangyueqian@pku.edu.cn)

◇ Task 1 - Chinese Grammatical Error Correction

Please fill out the Shared Task 1 Registration Form (Word File) and send it to the following registration email.

Registration Email: masr21@mails.tsinghua.edu.cn, C.C.liyinghu20@mails.tsinghua.edu.cn, zheng.haitao@sz.tsinghua.edu.cn

◇ Task 2 - Multi-perspective Scientific Machine Reading Comprehension

Please fill out the Shared Task 2 Registration Form (Word File) and send it to the following registration email.

Registration Email: 3220201024@bit.edu.cn, 3120201096@bit.edu.cn

◇ Task 3 - Math Word Problem Solving

Please fill out the Shared Task 3 Registration Form (Word File) and send it to the following registration email.

Registration Email: jzhanggr@connect.ust.hk, lei.wang.2019@phdcs.smu.edu.sg, yanwang.branden@gmail.com, shumingshi@tencent.com

◇ Task 4 - Conversational Aspect-based Sentiment Quadruple Analysis

Registration website: https://conasq.pages.dev/

Contact Email: lifei_csnlp@whu.edu.cn, boboli@whu.edu.cn

◇ Task 5 - Chinese Medical Instructional Video Question Answering

Registration website: https://cmivqa.github.io/

Please also fill out the Shared Task 5 Registration Form (Word File) and send it to the following registration email.

Registration Email: libincn@hnu.edu.cn, wengsyx@gmail.com, hu_guo@hnu.edu.cn

◇ Task 6 - Chinese Few-shot and Zero-shot Entity Linking

Please fill out the Shared Task 6 Registration Form (Word File) and send it to the following registration email.

Registration Email: xuzhenran@stu.hit.edu.cn

◇ Task 7 - Chinese Essay Discourse Coherence Evaluation

Please fill out the Shared Task 7 Registration Form (Word File) and send it to the following registration email.

Registration Email: why_hongyiwu@163.com

◇ Task 8 - Chinese Spelling Check

Please fill out the Shared Task 8 Registration Form (Word File) and send it to the following registration email.

Registration Email: xjyin@pku.edu.cn

◇ Task 9 - User Feedback Prediction and Response Generation

Please fill out the Shared Task 9 Registration Form (Word File) and send it to the following registration email.

Registration Email: dongshuang1@xiaomi.com

◇ Task 10 - Learn to Watch TV: Multimodal Dialogue Understanding and Response Prediction

Please fill out the Shared Task 10 Registration Form (Word File) and send it to the following registration email.

Registration Email: wangyueqian@pku.edu.cn

2023/03/15：announcement of shared tasks and call for participation;

2023/03/15：registration open;

2023/04/06：release of detailed task guidelines & training data;

2023/05/05：registration deadline;

2023/05/21：release of test data;

2023/05/31：participants’ results submission deadline;

2023/06/10：evaluation results release and call for system reports and conference paper;

2023/06/30：conference paper submission deadline (only for shared tasks);

2023/07/18：conference paper accept/reject notification;

2023/08/01：camera-ready paper submission deadline.

The evaluation papers are English only. The papers will be in the proceedings of the NLPCC-2023 conference (for English) which will be published as a volume in the Springer LNAI series (EI & ISTP indexed, English papers). Submissions should follow the LNCS formatting instructions. The maximum paper length is 12 pages, including references; The submissions must therefore be formatted in accordance with the standard Springer style sheets ([LaTeX][Microsoft Word]). All submissions should be prepared in the PDF format.

Manuscripts should be submitted electronically through the Softconf START conference management system https://www.softconf.com/nlpcc/Eval-2023. After logging in, please click the "make a new submission" button. Email submissions will not be accepted.

Yunbo CAO, Tencent

Shuang DONG, XiaoMi AI Lab

Hao FEI, National University of Singapore

Tao GE, Microsoft Research Asia

Hu GUO, Xiangya Hospital of Central South University

Baotian HU, Harbin Institute of Technology (Shenzhen)

Donghong JI, School of Cyber Science and Engineering, Wuhan University

Man LAN, East China Normal University

Bin LI, College of Electrical and Information Engineering, Hunan University

Bobo LI, School of Cyber Science and Engineering, Wuhan University

Fei LI, School of Cyber Science and Engineering, Wuhan University

Piji LI, Nanjing University of Aeronautics and Astronautics

Yinghui LI, Tsinghua Shenzhen International Graduate School, Tsinghua University

Wei LIU, XiaoMi AI Lab

Shirong MA, Tsinghua Shenzhen International Graduate School, Tsinghua University

Xianling MAO, School of Computer Science and Technology, Beijing Institute of Technology

Shaoguang MAO, Microsoft Research Asia

Yuxiang NIE, School of Computer Science and Technology, Beijing Institute of Technology

Zifei SHAN, Tencent

Xinshu SHEN, East China Normal University

Shuming SHI, Tencent AI Lab

Hanlin TENG, Renmin University of China

Lei WANG, Tencent AI Lab

Yan WANG, Tencent AI Lab

Xiaojun WAN, Beijing Founder Electronics Co., Ltd.

Yueqian WANG, Peking University

Yuxuan WANG, Peking University

Yixuan WENG, National Laboratory of Pattern Recognition Institute of Automation, CAS

Hongyi WU, East China Normal University

Yuanbin WU, East China Normal University

Yan XIA, Microsoft Research Asia

Zhenran XU, Harbin Institute of Technology (Shenzhen)

Rui YAN, Renmin University of China

Xunjian YIN, Peking University

Linlin YU, Founder Technology

Long YU, Founder Technology

Dan ZHANG, Beijing Founder Electronics Co., Ltd.

Jipeng ZHANG, Tencent AI Lab

Xiao ZHANG, School of Computer Science and Technology, Beijing Institute of Technology

Dongyan ZHAO, Peking University

Kang ZHAO, Renmin University of China

Haitao ZHENG, Tsinghua Shenzhen International Graduate School, Tsinghua University

Heqi ZHENG, School of Computer Science and Technology, Beijing Institute of Technology