ABSTRACT: Network Embedding is a process of learning low-dimensional representation vectors of nodes by comprehensively utilizing network characteristics. Besides structure properties, information networks also contain rich external information, such as texts and labels. However, most of the traditional learning methods do not consider this kind of information comprehensively, which leads to the lack of semantics of embeddings. In this paper, we propose a Semi-supervised Hierarchical Attention Network Embedding method, named as SHANE, which can incorporate external information in a semi-supervised manner. First, a hierarchical attention network is used to learn the text-based embeddings according to the content of nodes. Then, the text-based embeddings and the structure-based embeddings are integrated in a closed interaction way. After that, we further introduce the label information of nodes into the embedding learning, which can promote the nodes with the same label closed in the embedding space. Extensive experiments of link prediction and node classification are conducted on two real-world datasets, and the results demonstrate that our method outperforms other comparison methods in all cases.

ABSTRACT: Sarcasm detection research aims at determining whether a given text input contains sarcastic expressions. Detecting sarcasm is becoming an increasingly important task because expressions of sarcasm are widely used in subjective, user-generated, content,and existing models of social media analysis and sentiment prediction have difficulties handling sarcasm. Existing approaches to sarcasm detection are standard supervised learning models, often using deep neural networks, and they suffer from lack of sufficient training data. To address this problem, wepropose an adversarial learning framework built on convolutional neural network(CNN) and attention mechanism, which is trained from limited amounts of labeled data. This paper investigates two complementary adversarial learning approaches. In the first, by training with generated adversarial examples, we attempt to enhance the robustness and generalization ability of the classifier. In the second, we propose a domain transfer based adversarial learning approach, where the goal is to leverage cross-domain sarcasm data for improving the performance of sarcasm detection in the target domain. Experimental results on three sarcasm datasets show that: (1) both adversarial learning approaches proposed improve the performance of sarcasm detection, but the domain transfer based approach achieves higher performance; (2) combining the two proposed approaches further improves the performance of sarcasm detection.

ABSTRACT: Discourse structure analysis is an important task in Natural Language Processing (NLP) and it is helpful to many NLP tasks, such as automatic summarization and information extraction. However, there are only few researches on Chinese macro discourse structure analysis due to the lack of annotated corpora. In this paper, combining structure recognition with nuclearity recognition, we propose a Label Degeneracy Combination Model (LD-CM) to find the solution of structure recognition in the solution space of nuclearity recognition. Experimental results on the Macro Chinese Dis-course TreeBank (MCDTB) show that our model improves the accuracy by 1.21%, compared with the baseline system.

ABSTRACT: Word segmentation and part-of-speech tagging are two preliminary but fundamental components of Chinese natural language processing. With the upsurge of deep learning, end-to-end models are built without handcrafted features. In this work, we model Chinese word segmentation and part-of-speech tagging jointly on the basis of state-of-the-art BiRNN-CRF architecture. LSTM is adopted as the basic recurrent unit. Apart from utilizing pre-trained character embeddings and trigram features, we incorporate neural language model and conduct multi-task training. Highway layers are applied to tackle the discordance issue of the naive co-training. Experimental results on CTB5, CTB7, and PPD datasets show the effectiveness of the proposed method.

ABSTRACT: Chinese word segmentation (CWS) is an important task for Chinese NLP. Recently, many neural network based methods have been proposed for CWS. However, these methods require a large number of labeled sentences for model training, and usually cannot utilize the useful information in Chinese dictionary. In this paper, we propose two methods to exploit the dictionary information for CWS. The first one is based on pseudo labeled data generation, and the second one is based on multi-task learning. The experimental results on two benchmark datasets validate that our approach can effectively improve the performance of Chinese word segmentation, especially when training data is insufficient.

ABSTRACT: A deep learning model adaptive to both sentence-level and article-level paraphrase identification is proposed in this paper. It consists of pairwise unit similarity feature and semantic context correlation feature. In this model, sentences are represented by word and phrase embedding while articles are represented by sentence embedding. Those phrase and sentence embedding are learned from parse trees through Weighted Unfolding Recursive Auto-encoders (WURAE), an unsupervised learning algorithm. Then, unit similarity matrix is calculated by matching the pairwise lists of embedding. It is used to extract the pairwise unit similarity feature through CNN and k-max pooling layers. In addition, semantic context correlation feature is taken into account, which is captured by the combination of CNN and LSTM. CNN layers learn collocation information between adjacent units while LSTM extracts the long-term dependency feature of the text based on the output of CNN. This model is experimented on a famous English sentence paraphrase corpus, MSRPC, and a Chinese article paraphrase corpus. The results show that the deep semantic feature of text could be extracted based on WURAE, unit similarity and context correlation feature. We release our code of WURAE, deep learning model for paraphrase identification and pre-trained phrase end sentence embedding data for use by the community.

ABSTRACT: The Uyghur language has various inflectional affixes, complex structures and phonetic changes, which pose challenges for natural language processing tasks. In this research, we propose a collaborative analysis method for Uyghur morphology at character level. It includes three procedures: morpheme segmentation, morphological annotation and reduction of phonetic changes. The main characteristics of this method is to use a composite tag to represent the morpheme boundaries, annotations and phonetic changes. In addition, we use character sequence annotation to train the model. Experimental results show that the accuracy of morpheme segmentation, morphological annotation and reduction of phonetic reaches 95.86%, 92.39% and 99.70% respectively. The overall accuracy of the system reaches 91.84%.

ABSTRACT: Target extraction is an important task in target-based sentiment analysis, which aims at identifying the boundary of target in given text. Previous works mainly utilize conditional random field (CRF) with a lot of handcraft features to recognize the target. However, it is hard to manually extract effective features to boost the performance of CRF-based methods. In this paper, we employ gated recurrent units (GRU) with label inference, to find valid label path for word sequence. At the same time, we find that character-level features play important roles in target extraction, and represent each word by concatenating word embedding and character-level representations which are learned via character-level GRU. Further, we capture boundary features of each word from its context words by convolution neural networks to assist the identification of the target boundary, since the boundary of a target is highly related to its context words. Experiments on two datasets show that our model outperforms CRF-based approaches and demonstrate the effectiveness of features learned from character-level and context words.

ABSTRACT: This paper presents a novel neural architecture for aspect term extraction in fine-grained sentiment computing area. In addition to amalgamating sequential features (character embedding, word embedding and POS tagging information), we train an end-to-end Recurrent Neural Networks (RNNs) with meticulously designed dependency transmission between recurrent units, thereby making it possible to learn structural syntactic phenomena. The experimental results show that incorporating these shallow semantic features improves aspect term extraction performance compared to a system that uses no linguistic information, demonstrating the utility of morphological information and syntactic structures for capturing the affinity between aspect words and their contexts.

ABSTRACT: Aspect identification, a key subtask in Aspect-Based Sentiment Analysis (ABSA), aims to identify aspect categories from online user reviews. Inspired by the observation that different segments of a review usually express different aspect categories, we propose a reviews-segmentation-based method to improve aspect identification. Specifically, we divide a review into several segments according to the sentence structure, and then automatically transfer aspect labels from the original review to its derived segments. Trained with the new constructed segment-level dataset, a classifier can achieve better performance for aspect identification. Another contribution of this paper is extracting alignment features, which can be leveraged to further improve aspect identification. The experimental results show the effectiveness of our proposed method.

ABSTRACT: Natural Language Inference (NLI) is a fundamental and challenging task in Natural Language Processing (NLP). Most existing methods only apply one-pass forward inference process on a mixed matching feature, which is a concatenation of different matching features between a premise and a hypothesis. In this paper, we propose a new model called Multi-turn Inference Matching Network (MIMN) to perform multi-turn inference on different matching features. In each turn, the model focuses on one particular matching feature instead of the mixed matching feature. To enhance the interaction between different matching features, a memory component is employed to store the history inference information. The inference of each turn is performed on the current matching feature and the memory. We conduct experiments on three different NLI datasets. The experimental results show that our model outperforms or achieves the state-of-the-art performance on all the three datasets.

ABSTRACT: Due to the limited length and freely constructed sentence structures, it is a difficult classification task for short text classification, especially in multi-class classification. In this paper, an efficient meta learning framework is proposed for twitter classification. The tweets will be clustered into many sentence styles, which corresponding to new class labels. Thus, the original text classification task becomes few-shot learning task. When applying few-shot learning on benchmark datasets, our method Meta-CNN achieves improvement in accuracy and F1 scores on multi-class twitter classification, and outweigh some traditional machine learning methods and a few deep learning approaches.

ABSTRACT: Most of the previous emotion classifications are based on binary or ternary classifications, and the final emotion classification results contain only one type of emotion. There is little research on multi-emotional coexistence, which has certain limitations on the restoration of human's true emotions. Aiming at these deficiencies, this paper proposes a Bidirectional Long-Short Term Memory Multiple Classifiers (BLSTM-MC) model to study the five classification problems in code-switching text, and obtains text contextual relations through BLSTM-MC model. It fully considers the relationship between different emotions in a single post, at the same time, the Attention mechanism is introduced to find the importance of different features and predict all emotions expressed by each post. The model achieved third place in all submissions in the conference NLP&&CC_task1 2018.

ABSTRACT: This paper describes the methods for the DeepIntell who participated the task1 in the NLPCC2018. The task1 is to label the emotion in a code-switching text. Note that, there may be more than one emotion in a post in this task. Hence, the assessment task is a multilabel classification task. At the same time, the post contains more than one language, and the emotion can be expressed by either monolingual or bilingual form. In this paper, we propose a novel method of converting multi-label classification into binary classification task and ensemble learning for code-switching text with sampling and emotion lexicon. Experiments show that the proposed method has achieved better performance in the code-switching text task.

ABSTRACT: In the multi-label classification task (Automatic Tagging of Zhihu Questions), we present a classification system which includes five processes. Firstly, we use a preprocessing step to solve the problem that there is too much noise in the training dataset. Secondly, we choose several neural network models which proved effective in text classification task. Then we introduce k-max pooling structure to these models to fit this task. Thirdly, in order to obtain a better performance in ensemble process, we use an experiment-designing process to obtain classification results that are not similar to each other and all achieve relatively high scores. Fourthly, we use an ensemble process. Finally, we propose a method to estimate how many labels should be chosen. With these processes, our F1 score achieves 0.5194, which ranked No. 3.

ABSTRACT: This paper introduces the Alibaba NLP team’s system for NLPCC 2018 shared task of Chinese Grammatical Error Correction (GEC). Chinese as a Second Language (CSL) learners can use this system to correct grammatical errors in texts they wrote. We proposed a method to combine statistical and neural models for the GEC task. This method consists of two modules: the correction module and the combination module. In the correction module, two statistical models and one neural model generate correction candidates for each input sentence. Those two statistical models are a rule-based model and a statistical machine translation (SMT) -based model. The neural model is a neural machine translation (NMT) -based model. In the combination module, we implemented it in a hierarchical manner. We first combined models at a lower level, which means we trained several models with different configurations and combined them. Then we combined those two statistical models and a neural model at the higher level. Our system reached the second place on the leaderboard released by the official.

ABSTRACT: The NLPCC 2018 Chinese Grammatical Error Correction (CGEC) shared task seeks the best solution to detecting and correcting grammatical errors in Chinese essays written by non-native Chinese speakers. This paper describes Youdao NLP team’s approach to this challenge, which won the 1st place in the contest. Overall, we cast the problem as a machine translation task. We use a staged approach and design specific modules targeting at particular errors, including spelling, grammatical, etc. The task uses M2 Scorer [5] to evaluate every system’s performance, and our final solution achieves the highest recall and F0:5.

ABSTRACT: Recently sequence-to-sequence (Seq2Seq) model and its variants are widely used in multiple summarization tasks e.g., sentence compression, headline generation, single document summarization, and have achieved significant performance. However, most of the existing models for abstractive summarization suffer from some undesirable shortcomings such as generating inaccurate contents or insufficient summary. To alleviate the problem, we propose a novel approach to improve the summary’s informativeness by explicitly incorporating topical keywords information from the original document into a pointer-generator network via a new attention mechanism so that a topic-oriented summary can be generated in a context-aware manner with guidance. Preliminary experimental results on the NLPCC 2018 Chinese document summarization benchmark dataset have demonstrated the effectiveness and superiority of our approach. We have achieved significant performance close to that of the best performing system in all the participating systems.

ABSTRACT: We present Summary++, the model that competed in NLPCC2018’s Summary task. In this paper, we describe in detail of the task, our model, the results and other aspects during our experiments. The task is News article summarization in Chinese, where one sentence is generated per article. We use a neural encoder decoder attention model with pointer generator network, and modify it to focus on words attented to rather than words predicted. Our model archive second place in the task with a score of 0.285. The highlights of our model is that it run at character level, no extra features (e.g. part of speech, dependency structure) were used and very little preprocessing were done.

ABSTRACT: Social networking sites have been growing at an unprecedented rate in recent years. User profiling and personalized recommendation plays an important role in social networking, such as targeting advertisement and personalized news feed. For NLPCC Task 8, there are two subtasks. Subtask one is User Tags Prediction (UTP), which is to predict tags related to a user. We consider UTP as a Multi Label Classification (MLC) problem and proposed a CNN-RNN framework to explicitly exploit the label dependencies. The proposed framework employs CNN to get the user profile representation and the RNN module captures the dependencies among labels. Subtask two, User Following Recommendation (UFR), is to recommend friends to the users. There are mainly two approaches: Collaborative Filtering (CF) and Most Popular Friends (MPF), and we adopted a combination of both. Our experiments show that both of our methods yield clear improvements in F1@K compared to other algorithms and achieved first place in both subtasks.

SPEAKER BIO: 张岳，博士，西湖大学副教授，之前担任新加坡科技设计大学的助理教授。在2012年7月加入新加坡科技设计大学之前，在剑桥大学从事博士后工作。张岳博士分别在清华大学获得学士学位、在牛津大学获得硕士学位和博士学位。他的研究兴趣包括：自然语言处理，机器学习，人工智能。他一直致力于统计句法分析、文本合成、机器翻译、情感分析和股票市场分析。
ABSTRACT: 在这个报告中，简要介绍三个实际问题驱动的研究。首先，自然语言处理有很多不同领域、不同问题的语料。为了能够使用尽可能多地人工标注解决每一个问题，我们研究神经网络结构来，最大限度利用有信息重叠的数据。第二，双向循环神经网络，虽然有效但是速度成为应用平静。我们研究如何换一种方式实现编码系统，提高运行速度。第三，中文很多问题需要事先分词，但是分词错误可能会蔓延到下游任务。我们试图使用词汇表保留分词歧义，但同时利用词汇信息。

SPEAKER BIO: 程勇，从清华大学交叉信息研究院获得博士学位，现在腾讯AI平台部担任高级研究员，主要研究领域为机器翻译，已在人工智能顶级学术会议(如ACL, IJCAI，AAAI等）上发表多篇论文。
ABSTRACT: 由于神经机器翻译是一个端到端的翻译系统，其对于输入中的微小扰动极其敏感。例如，将输入中某个词替换成其近义词，会导致输出结果发生剧烈变化，甚至修改翻译结果的极性。本文提出了对抗性稳定训练准则来同时增强神经机器翻译的编码器与解码器的鲁棒性。给定一个输入句子x，我们首先生成与其对应的扰动输入x’，接着采用对抗训练鼓励编码器对于x和x’生成相似的中间表示，同时要求解码器端输出相同的目标句子y。本文中，我们提出了两种构造扰动输入的方法，第一种在特征级别（词向量）中加入高斯噪声，第二种在词级别中用近义词来替换原词，我们的框架可以扩展到更多得噪声扰动方法。实验表明我们的方法可以同时增强神经机器翻译模型的鲁棒性和翻译质量。

SPEAKER BIO: 林衍凯，清华大学计算机系博士生五年级，来自清华大学自然语言处理组， 由孙茂松教授和刘知远助理教授共同指导，主要研究方向为知识图谱表示、构建和应用。目前已在人工智能、自然语言处理等领域的著名国际会议IJCAI，AAAI，EMNLP，ACL发表相关论文多篇，Google Scholar引用数超过600。
ABSTRACT: 近年来，如何在自然语言理解任务中利用大规模知识图谱如DBpedia、Freebase、Wikidata等的信息来提升相关任务的效果已经成为一个研究热点。在其中，如何在深度学习框架中对知识图谱进行表示、丰富及有机融合已经关键的挑战。在这次报告中，讲者将重点介绍他在知识表示、获取以及应用中的相关工作，并探讨在研究过程的问题。

SPEAKER BIO: 刘一佳，哈工大在读博士，美国华盛顿大学访问学生。他的研究兴趣包括中文分词、句法分析、对话系统等。博士期间在ACL、IJCAI等自然语言处理、人工智能的顶级重要会议上发表论文8篇，发表专利一项。参与开发的语言技术平台被500余家公司科研机构使用。2016年，参与项目获黑龙江省科学进步一等奖。2018年，参与团队在CoNLL 2018国际多语言通用依存分析评测中夺得冠军。
ABSTRACT: 在这项工作中，我们研究了面向任务的对话系统中语言理解模块的数据增广问题。我们利用训练数据中一对句子的语义相近性，提出了基于序列到序列生成的数据增广框架。同时，我们将多样性等级融入到生成过程中，从而获得更多样化的数据增广结果。在航空以及斯坦福多轮对话的小规模数据集上，我们的数据增广方法取得了6.4与10.0的性能提升。
这项工作是与一位低年级博士合作完成的。我将基于这次合作过程，结合博士期间的经验教训，对于开展科研工作，论文写作以及更好的合作分享一些粗浅的观点。

ABSTRACT: Relation classification is an important task in natural language processing (NLP) fields. State-of-the-art methods are mainly based on deep neural networks. This paper proposes a bi-channel tree convolution based neural network model, BiTCNN, which combines syntactic tree features and other lexical level features together in a deeper manner for relation classification. First, each input sentence is parsed into a syntactic tree. Then, this tree is decomposed into two sub-tree sequences with top-down decomposition strategy and bottom-up decomposition strategy. Each sub-tree represents a suitable semantic fragment in the input sentence and is converted into a real-valued vector. Then these vectors are fed into a bi-channel convolutional neural network model and the convolution operations re performed on them. Finally, the outputs of the bi-channel convolution operations are combined together and fed into a series of linear transformation operations to get the final relation classification result. Our method integrates syntactic tree features and convolution neural network architecture together and elaborates their advantages fully. The proposed method is evaluated on the SemEval 2010 data set. Extensive experiments show that our method achieves better relation classification results compared with other state-of-the-art methods.

ABSTRACT: Named Entity Recognition (NER) is the preliminary task in many basic NLP technologies and deep neural networks has shown their promising opportunities in NER task. However, the NER tasks covered in previous work are relatively simple, focusing on classic entity categories (Persons, Locations, Organizations) and failing to meet the requirements of newly-emerging application scenarios, where there exist more informal entity categories or even hierarchical category structures. In this paper, we propose a multi-task learning based subtask learning strategy to combat the complexity of modern NER tasks. We conduct experiments on a complex Chinese NER task, and the experimental results demonstrate the effectiveness of our approach.

ABSTRACT: Identifying entity boundaries and eliminating entity ambiguity are two major challenges faced by Chinese named entity recognition researches. This paper proposes a five-stroke based CNN-BiRNN-CRF network for Chinese named entity recognition. In terms of input embeddings, we apply five-stroke input method to obtain stroke-level representations, which are concatenated with pre-trained character embeddings, in order to explore the morphological and semantic information of characters. Moreover, the convolution neural network is used to extract n-gram features, without involving hand-crafted features or domain-specific knowledge. The proposed model is evaluated and compared with the state-of-the-art results on the third SIGHAN bakeoff corpora. The experimental results show that our model achieves 91.67% and 90.68% F1-score on MSRA corpus and CityU corpus separately.

ABSTRACT: Distant supervised relation extraction is an efficient method to find novel relational facts from very large corpora without expensive manual annotation. However, distant supervision will inevitably lead to wrong label problem, and these noisy labels will substantially hurt the performance of relation extraction. Existing methods used multi-instance learning and selective attention to reduce noise effect. However, they usually cannot fully utilize the supervision information and cannot fully eliminate the effect of noise. In this paper, we propose a method called Neural Instance Selector (NIS) to solve these problems. Our approach contains three modules, a sentence encoder to encode input texts into hidden vector representations, a NIS module to filter the less informative sentences via multilayer perceptrons and logistic classification, and a selective attention module to select the important sentences. Experiment results show that our method can effectively filter noisy data and achieve better performance than existing methods.

ABSTRACT: As one of the important tasks in text comprehension, the identification of discourse relations is the base of downstream applications such as Q&A system. And the implicit discourse relations identification, referring to the case without any conjunction indicating discourse relations in text units, is to infer discourse relation by analyzing the semantic between two text units. With the implicit discourse relation identification, there will be obviously more difficulties than the explicit one. The existing studies mainly focus on the English recognition of coarse-grained implicit discourse relation, and there is no specific research on the fine-grained implicit discourse. In fact, identifying the fine-grained implicit discourse relation can obtain the logical semantic role of the corresponding text unit in the logical relationship, which is more valuable for practical applications. Aiming at the identification of Chinese fine-grained implicit discourse relation and taking the directionality characteristic in account, this paper therefore propose a feature learning algorithm based on the distant supervision to label explicit discourse data automatically. The relative position information between conjunction and words are applied to train the intensive word representation. Then the rhetorical function of words and the directionality of relations are encoded into the representation of intensive words, which will be applied to the relation classification of fine-grained implicit discourses. From the experimental studies of our proposed approach, the classification accuracy reaches 56.21% and F1-value reaches 51.23%, which are better than those approaches neglecting the directionality of discourse relations.

ABSTRACT: Machine reading comprehension is a task to model relationship between passage and query. In terms of deep learning framework, most of state-of-the-art models simply concatenate word and character level representations, which has been shown suboptimal for the concerned task. In this paper, we empirically explore different integration strategies of word and character embeddings and propose a character-augmented reader which attends character-level representation to augment word embedding with a short list to improve word representations, especially for rare words. Experimental results show that the proposed approach helps the baseline model significantly outperform state-of-the-art baselines on various public benchmarks.

ABSTRACT: In this paper, we introduce Adversarial-and-attention Network (A3Net) for Machine Reading Comprehension. This model extends existing approaches from two perspectives. First, adversarial training is applied to several target variables within the model, rather than only to the inputs or embeddings. We control the norm of adversarial perturbations according to the norm of original target variables, so that we can jointly add perturbations to several target variables during training. As an effective regularization method, adversarial training improves robustness and generalization of our model. Second, we propose a multi-layer attention network utilizing three kinds of high-efficiency attention mechanisms. Multi-layer attention conducts interaction between question and passage within each layer, which contributes to reasonable representation and understanding of the model. Combining these two contributions, we enhance the diversity of dataset and the information extracting ability of the model at the same time. Meanwhile, we construct A3Net for the WebQA dataset. Results show that our model outperforms the state-of-the-art models (improving Fuzzy Score from 73.50% to 77.0%).

ABSTRACT: Existing works on machine reading comprehension mostly focus on extracting text spans from passages with the assumption that the passage must contain the answer to the question. This assumption usually cannot be satisfied in real-life applications. In this paper, we study the reading comprehension task in which whether the given passage contains the answer is not specified in advance. The system needs to correctly refuse to give an answer when a passage does not contain the answer. We develop several baselines including the answer extraction based method and the passage triggering based method to address this task. Furthermore, we propose an answer validation model that first extracts the answer and then validates whether it is correct. To evaluate these methods, we build a dataset SQuAD-T based on the SQuAD dataset, which consists of questions in the SQuAD dataset and includes relevant passages that may not contain the answer. We report results on this dataset and provides comparisons and analysis of the different models.

ABSTRACT: Human agents in technical customer support provide users with instructional answers to solve a task. Developing a technical support question answering (QA) system is challenging due to the broad variety of user intents. Moreover, user questions are noisy (for example, spelling mistakes), redundant and have various natural language expresses, which are challenges for QA system to match user queries to corresponding standard QA pair. In this work, we combine question intent categories classification and semantic matching model to filter and select correct answers from a back-end knowledge base. Using a real world user chatlog dataset with 60 intent categories, we observe that while supervised models, perform well on the individual classification tasks. For semantic matching, we add muti-info (answer and product information) into standard question and emphasize context information of user query (captured by GRU) into our model. Experiment results indicate that neural multi-perspective sentence similarity networks outperform baseline models. The precision of semantic matching model is 85%.

ABSTRACT: Grapheme to phoneme (G2P) conversion is the assignment of converting word to its pronunciation. It has important applications in text-to-speech (TTS), speech recognition and sounds-like queries in textual databases. In this paper, we presents the first application of sequence-to-sequence (Seq2Seq) Long Short-Term Memory(LSTM) model with the attention mechanism for Mongolian G2P conversion, Furthermore, we propose a novel hybrid approach of combining rules with Seq2Seq LSTM model for Mongolian G2P conversion, and implement the Mongolian G2P conversion system. The experimental results show that: Adopting Seq2Seq LSTM model can obtain better performance than the traditional methods in Mongolian G2P conversion, and the hybrid approach further improves G2P conversion performance. The word error rate (WER) relatively reduces by 10.8% and the phoneme error rate (PER) approximately reduces by 1.6% through comparing with the Mongolian G2P conversion method being used based on the joint-sequence models, which completely meets the practical requirements of Mongolian G2P conversion.

ABSTRACT: Non-Factoid question is an important topic in the research of question answering system. A number of methods have been proposed for general domain problems. However, these methods are less effective for non-factoid questions for special domains such as financial and taxation. Without deep semantic analysis for complex non-factoid questions, it is difficult to utilize domain knowledge in question understanding and answer extraction. In this research, a semantic-based retrieval method was proposed to extract answer sentences from tax regulation and cases. Firstly, a domain knowledge base was employed to generate semantic annotations for questions, regulations and cases. Secondly, a filtering system was developed for the removal of irrelevant cases from answer candidates. In addition, a semantic similarity measurement method was employed for answer extraction. Finally, a rank model was proposed for the optimization of the retrieved results. In order to validate the proposed method, a series of experiments were performed on real-life dataset. Experiment results show noticeable improvement in accuracy and performance.

ABSTRACT: Most of question-answer pairs in question answering task are generated manually, which is inefficient and expensive. However, the existing work on automatic question generation is not good enough to replace manual annotation. This paper presents a system to generate questions from a knowledge base in Chinese. The contribution of our work contains two parts. First we offer a neural generation approach using long short term memory (LSTM). Second, we design a new format of input sequence for the system, which promotes the performance of the model. On the evaluation of KBQG of NLPCC 2018 Shared Task 7, our system achieved 73.73 BLEU, and took the first place in the evaluation.

ABSTRACT: This report analyzes the problem of spoken language understanding, how the problem is simplified in the NLPCC shared task, and the properties of the official datasets. It also describes the system we developed for the shared task and provides experimental analysis that explains how promising results could be achieved by careful usage of standard machine learning and natural language processing techniques and external resources.

ABSTRACT: In task-oriented dialogue systems, spoken language understanding (SLU) aims to convert users’ queries expressed by natural language to structured representations. SLU usually consists of two parts, namely intent identification and slot filling. Although many methods have been proposed for SLU, these methods generally process each utterance individually, which loses context information in dialogues. In this paper, we propose a hierarchical LSTM based model for SLU. The dialogue history is memorized by a turn-level LSTM and it is used to assist the prediction of intent and slot tags. Consequently, the understanding of the current turn is dependent on the preceding turns. We conduct experiments on the NLPCC 2018 Shared Task 4 dataset. The results demonstrate that the dialogue history is effective for SLU and our model outperforms all baselines.

ABSTRACT: This paper describes the system we submitted to Task 5 in NLPCC 2018, i.e., Multi-Turn Dialogue System in Open-Domain3. This work focuses on the second subtask: Retrieval Dialogue System. Given conversation sessions and 10 candidates for each dialogue session, this task is to select the most appropriate response from candidates. We design a memory-based matching network integrating sequential matching network and several NLP features together to address this task. Our system finally achieves the precision of 62:61% on test set of NLPCC 2018 subtask 2 and officially released results show that our system ranks 1st among all the participants

ABSTRACT: This paper describes our method for sub-task 2 of Task 5: multi-turn conversation retrieval, in NLPCC2018. Given a context and some candidate responses, the task is to choose the most reasonable response for the context. It can be regarded as a matching problem. To address this task, we propose a deep neural model named RCMN which focus on modeling relevance consistency of conversations. In addition, we adopt one existing deep learning model which is advanced for multi-turn response selection. And we propose an ensemble strategy for the two models. Experiments show that RCMN has good performance, and ensemble of two models makes good improvement. The official results show that our solution takes 2nd place. We open the source of our code on GitHub, so that other researchers can reproduce easily.

ABSTRACT: Machine Reading Comprehension (MRC) has long been a central problem in Natural Language Processing (NLP). 2018 NLP Challenge on Machine Reading Comprehension provides a large-scale application-oriented dataset for Chinese Machine Reading Comprehension, which is much more challenging than previous Chinese MRC dataset. To cope with those challenges, we put effort in all aspects, including preprocessing strategy, feature expression, model design, loss function and training criterion. Our system achieves 63.38 in ROUGE-L score and 59.23 in BLEU-4 score on the final test set, ranked first among 105 participating teams.

ABSTRACT: Machine reading comprehension (MRC) is an important task in natural language processing and artificial intelligence. It has received wide attention across academia and industry in recent years. To improve the processing ability of a MRC system on multi-document Chinese datasets, here we propose N-Reader, an end-to-end neural network based MRC model. It uses a two-layer self-attention mechanism to encode the input documents. With such mechanism, both the information from a single document and the similarity information from multiple documents can be utilized. Besides, we also propose a multi-paragraph completion algorithm to preprocess the input documents. This preprocessing method can further help the model to recognize the semantics-related paragraphs among input documents, based on which a better answer sequence could be obtained. Using this model, we participated in the “2018 NLP Challenge on Machine Reading Comprehension” which is organized jointly by Chinese Information Processing Society of China (CIPS), Chinese Computer Federation (CCF), and Baidu Inc.. Finally, our model ranks No.3 among the hugely competitive models.

ABSTRACT: For machine reading of real scenes, it is important to understand the complex information presented by the text. For Chinese machine reading tasks of continuous answer span in multi-passage, we proposes a model based on deep hierachical features to extract the three-level deep features of details, snippets, and full-texts, so as to grasp the information contained in the passages from multiple perspectives. In this model, words are represented by word vectors, and after being coded in a recurrent layer, detailed features are obtained, and snippets features are constructed through several convolution layers and highway layers. Moreover, full-text features are extracted from candidate passages to perform the overall inspection. Finally, through these features, the passage where the answer is located and the answer spans within the passage is determined. In 2018 NLP Challenge on Machine Reading Comprehension, the single model achieved a Rouge-L score of 57.55 and a Bleu-4 score of 50.87, which proved the good result.

ABSTRACT: Internet and the rapid growth of data scale, how to apply the machine reading comprehension technology to parsing massive unstructured data to help users find satisfactory answers quickly and accurately is a popular research orientation in the field of natural language understanding. By exploiting the deep neural network model in machine reading comprehension, we construct the RBiDAF model. Firstly, by the data exploration to the DuReader dataset and the preprocessing of the data, the features that are constructive to model training are extracted. Then, based on the BiDAF model, a machine reading comprehension model for multi-document reordering is proposed, named RBiDAF. This model adds a paragraph-Ranking-layer to the four-layer standard BiDAF model, where we propose the ParaRanking algorithm with multi-feature fusion in the paragraph-Ranking-layer. Additionally, in order to predict comprehensive answer, we propose the multi-answer cross validation algorithm based on prior knowledge. Finally, the RBiDAF model has shown good results in the final evaluation in the 2018 Machine Reading Comprehension Technology Competition..

SPEAKER BIO: 肖桐，博士，东北大学计算机科学与工程学院副教授，中国中文信息学会首届优秀博士论文提名奖获得者，NiuTrans团队技术负责人。2012年博士毕业于东北大学，并先后在日本富士施乐公司、微软亚洲研究院进行访问学习。2013-2014赴英国剑桥大学开展博士后研究。作为技术负责人主持NiuTrans开源机器翻译系统的研发，并在国内外机器翻译评测WMT、CWMT、NTCIR Patent MT中多次取得第一及第二的成绩。2016获得钱伟长中文信息科学技术奖一等奖。从事机器翻译、语言建模方面的研究。在人工智能及自然处理语言领域重要期刊及会议AI、JAIR、TALIP、AAAI、ACL、EMNLP、COLING发表论文20余篇。社会学术兼职包括：中国中文信息学会青年工作委员会委员、中国中文信息学会信息检索与内容安全专业委员会委员等。
ABSTRACT: 在本报告中，报告人将从自身从事科研的成长经历出发，分享一些他在从事机器翻译研究中所遇到的困难和体会，没有太多成功的范例，更多的是失败后总结的经验，内容涉及：选题、论文写作、系统研发等等。同时，报告人也会分享最近几年他在实验室建设方面的心得和对研究生同学的一些建议。

SPEAKER BIO: 王少楠，中科院自动化研究所博士生，于2018年6月博士毕业并在自动化所担任助理研究员。主要研究方向包括文本表示模型，语义分析与计算等。已在人工智能顶级学术会议发表多篇论文。
ABSTRACT: 文本表示指通过某种方式将自然语言文本编码为计算机可以处理的形式，这 是实现自然语言理解最基础也是最重要的步骤。高质量的文本表示可以使计算机 有效地完成各种自然语言相关的任务，如机器翻译、自动问答、人机对话等，因此 开展这项研究具有重要的理论意义和应用价值。
对文本表示模型来说，将不同类型信息进行有效地融合对获取高质量文本表示至关重要。本文围绕如何设计有效的信息融合方法来学习高质量的文本表示展开，重点关注三种类型信息的融合方法:词汇表示中多种模态信息的融合、句子表示中底层词汇信息的融合、以及句子表示中字符与词汇信息的融合。在多个数据集上的实验表明，所提出的方法可以有效的提升词汇和句子表示学习的效果。

SPEAKER BIO: 周浩，字节跳动人工智能实验室研究员，2017年博士毕业于南京大学，研究方向主要包括句法分析，机器翻译等。已在自然语言理解顶级学术会议和期刊，如 ACL，EMNLP，TACL等发表多篇论文。
ABSTRACT: Existing neural machine translation systems do not explicitly model what has been translated and what has not during the decoding phase. To address this problem, we propose a novel mechanism that separates the source information into two parts: translated Past contents and untranslated Future contents, which are modeled by two additional recurrent layers. The Past and Future contents are fed to both the attention model and the decoder states, which offers NMT systems the knowledge of translated and untranslated contents. Experimental results show that the proposed approach significantly improves translation performance in Chinese-English, German-English and English-German translation tasks. Specifically, the proposed model outperforms the conventional coverage model in both of the translation quality and the alignment error rate.

SPEAKER BIO: Yichuan Hu is cofounder and CTO of Laiye (来也). Laiye focuses on conversational technologies and Laiye’s mission is to empower everyone with intelligent assistant. Previously, Yichuan cofounded personalized video recommendation engine Jinwankansha (今晚看啥) which was acquired by Baidu in 2012. Yichuan received his B.S. and M.S. degrees from Tsinghua University, Ph.D. degree from the University of Pennsylvania.
ABSTRACT: Chabot enables end users to communicate with an agent in a more natural and personalized way, and it has been widely used in our daily life. At Laiye, we built the most popular chatbot on WeChat that helps users get a variety of things done through conversation, such as calendar management, taxi hailing, etc. In addition, Laiye offers intelligent conversation solutions Wulai (吾来) for enterprises in different industries like retail, education, finance, etc. In this talk, we will present the following topics: 1. The architecture of a chatbot; 2. The use of deep learning in dialog system; 3. Typical applications and real world examples of chatbots.

ABSTRACT: In this paper, we address the problem of learning better word representations for neural machine translation (NMT). We propose a novel approach to NMT model training based on coarse-to-fine learning paradigm, which is able to infer better NMT model parameters for a wide range of less-frequent words in the vocabulary. To this end, our proposed method first groups source and target words into a set of hierarchical clusters, then a sequence of NMT models are learned based on it with growing cluster granularity. Each subsequent model inherits model parameters from its previous one and refines them with finer-grained word-cluster mapping. Experimental results on public data sets demonstrate that our proposed method significantly outperforms baseline attention-based NMT model on Chinese-English and English-French translation tasks.

ABSTRACT: Although neural machine translation(NMT) yields promising translation performance, it unfortunately suffers from over- and under-translation issues, of which studies have become research hotspots in NMT. At present, these studies mainly apply the dominant automatic evaluation metrics, such as BLEU, to evaluate the overall translation quality with respect to both adequacy and fluency. However, they are unable to accurately measure the ability of NMT systems in dealing with the above-mentioned issues. In this paper, we propose two quantitative metrics, the Otem and Utem, to automatically evaluate the system performance in terms of over- and under-translation respectively. Both metrics are based on the proportion of mismatched n-grams between gold reference and system translation. We evaluate both metrics by comparing their scores with human evaluations, where the values of Pearson Correlation Coefficient reveal their strong correlation. Moreover, in-depth analyses on various translation systems indicate some inconsistency between BLEU and our proposed metrics, highlighting the necessity and significance of our metrics.

ABSTRACT: Sequential word encoding lacks explicit representations of structural dependencies (e.g. tree, segment) over the source words in neural machine translation. Instead of using source syntax, in this paper we propose a source segment encoding (SEE) approach to modeling source segments in encoding process by two methods. One is to encode off-the-shelf n-grams of the source sentence into original source memory. The other is to jointly learn an optimal segmentation model with the translation model in an end-to-end manner without any supervision of segmentation. Experimental results show that the SEE method yields an improvement of 2.1+ BLEU points over the baselines on the Chinese-English translation task.

ABSTRACT: Chinese phonologic features play an important role not only in the sentence pronunciation but also in the construction of a native Chinese sentence. To improve the machine translation performance, in this paper we propose a novel phonology-aware neural machine translation (PA-NMT) model where Chinese phonologic features are leveraged for translation tasks with Chinese as the target. A separate recurrent neural network (RNN) is constructed in NMT framework to exploit Chinese phonologic features to help facilitate the generation of more native Chinese expressions. We conduct experiments on two translation tasks: English-to-Chinese and Japanese-to-Chinese tasks. Experimental results show that the proposed method significantly outperforms state-of-the-art baselines on these two tasks.

ABSTRACT: To better deal with Chinese zero elements, this paper makes a theoretical analysis from discourse perspective and completes the construction of the Chinese Discourse Zero Corpus (CDZC). First, the necessity of corpus construction has been explored based on the research of existing theoretical and data sources. Then, the top-dowm and forword search annotation strategy and the combination of the human machine is used to complete corpus annotation. finally, the detailed statistics analysis shows that CDZC can fully reflect the characters of Chinese linguistic and provide corpus resources for related research.

ABSTRACT: In the literature, various supervised learning approaches have been adopted to address the task of emotion classification. However, the performance of these approaches greatly suffers when the size of the labeled data is limited. In this paper, we tackle this challenge from a cross-lingual sensoria where the labeled data in a resource-rich language (i.e., English in this study) is employed to improve the emotion classification performance in a resource-poor language (i.e., Chinese in this study). Specifically, we first use machine translation services to eliminate the language gap between Chinese and English data and then propose a joint learning framework to leverage both Chinese and English data, which develops auxiliary representations from several auxiliary emotion classification tasks. Furthermore, in our joint learning approach, we introduce an attention mechanism to capture informative words. Empirical studies demonstrate the effectiveness of the proposed approach to emotion classification.

ABSTRACT: Aspect-level sentiment classification aims to determine the sentiment polarity of the sentence towards the aspect. The key element of this task is to characterize the relationship between the aspect and the contexts. Some recent attention-based neural network methods regard the aspect as the attention calculation goal, so they can learn the association between aspect and contexts directly. However, the above attention model simply uses the word embedding to represent the aspect, it fails to make a further improvement on the performance of aspect sentiment classification. To solve this problem, this paper proposes a dependency subtree attention network (DSAN) model. The DSAN model firstly extracts the dependency subtree that contains the descriptive information of the aspect based on the dependency tree of the sentence, and then utilizes a bidirectional GRU network to generate an accurate aspect representation, and uses the dot-product attention function for the dependency subtree aspect representation, which finally yields the appropriate attention weights. The experimental results on SemEval 2014 Datasets demonstrate the effectiveness of the DSAN model.

ABSTRACT: Recently, classifying sentiment polarities or emotion categories of social media text has drawn extensive attentions from both academic and industrial communities. However, limited efforts have been paid for emotion intensity prediction problem. In this paper, we propose a novel attention mechanism for CNN model that associates attention based weights for every convolution window. Furthermore, a new activation function is incorporated into the full-connected layer, which can alleviate the small gradient problem in function’s saturated region. Experiment results on benchmark dataset show that our pro-posed model outperforms several strong baselines and achieves competitive performance with the state-of-the-art non-ensemble models. Unlike the reported model used different neural network architectures for different emotion categories, our proposed model utilizes a unified architecture for intensity prediction.

ABSTRACT: In e-commerce systems, users’ ratings play an important role in many scenarios such as reputation and trust mechanisms and recommender systems. A general assumption in these techniques is that users’ ratings represent their true feelings. Although it has long been adopted in previous work, this assumption is not necessarily true.
In this paper, we first present an in-depth study of the inconsistency between users’ ratings and their reviews. Then we propose an approach to mine users’ "true ratings", which better represent their real feelings, from textual reviews based on Gated Recurrent Unit (GRU) and hierarchical attention techniques. One major contribution is that we are about the first, to the best of our knowledge, to investigate this new problem of discovering users’ true ratings, and to provide direct solutions to revise ratings that are insincere and inconsistent.
Comparative experiments on a real e-commerce dataset have been conducted, which show that the "true ratings" learned by the proposed model is significantly better than the original ones in terms of consistency with the reviews in three sets of crowdsourcing-based evaluations. Furthermore, leveraging different state-of-art recommendation approaches based on the learned "true ratings", more effective results have been achieved at all times in rating prediction task.

ABSTRACT: Most of existing cross-domain sentiment classification methods are not expressive enough to capture rich representation of text, and class noise accumulated during transfer process can lead to negative transfer which can adversely affect performance. To address these issue, this paper propose a method combining textual representation learning and transfer learning algorithm for cross-domain sentiment classification. This method first builds a hierarchical attention network to generate document representations with local semantic information. Afterwards, the authors utilize the class noise estimation algorithm to detect the negative transfer samples in transferred samples and remove them. Finally, the sentiment classifier is trained on the expanded dataset from samples in target domain and transferred ones in source domain. Compared with the baselines, two experiments on large-scale product review datasets show that the method is able to effectively reduce the RMSE of cross-domain sentiment classification by 1.5% and 1.0% respectively.

ABSTRACT: The research of sentiment analysis has a great significance for public opinion monitoring, personalized recommendation, or social computing. This paper mainly proposes a text sentiment analysis method based on representation learning. Firstly, an improved training model based on C&W model is proposed which can integrate emotional information and part of speech information in the training process of word embedding; and then using the evaluation of data sets of NLP&CC'2013 to compare experimental results with different models. The experimental results show that the C&W-SP model which combines emotion information and part of speech information has the best performance and confirm the advanced nature of the proposed method.

ABSTRACT: The purpose of the work is to research the error patterns of production and perception of Mandarin monosyllabic tone by college students from Amdo Tibetan rural and pastoral areas, and make the analysis of the causes of acoustic in both errors. We do the work through the two experiments of perception and production of tone. We use the methods of combining the speech engineering and experimental phonetics. Results show that the error rate of tone perception is highly correlated [r=0.92] with that of tone production. The level of Mandarin in Amdo Tibetan rural area is higher than that in pastoral area both in terms of tone perception and production. The hierarchy of difficulty for the four grades in rural area is as follows: sophomore > freshman > junior > senior, pastoral area is as follows: freshman > sophomore > junior > senior. The hierarchy of difficulty for the four tones both in rural and pastoral areas is as follows: Tone 2 >Tone 3 > Tone 1 >Tone 4.Tone 2 and 3 are most likely to be confused. There is no obvious tone shape bias of the four tones, but the tone domain is narrow and the location of the tone domain is lower than standard Mandarin both in rural and pastoral areas.

ABSTRACT: Emotional intelligence is one of the key parts of human intelligence. Exploring how to endow conversation models with emotional intelligence is a recent research hotspot. Although several emotional conversation approaches have been introduced, none of these methods were able to decide an appropriate emotion category for the response. We propose a new neural conversation model which is able to produce reasonable emotion interaction and generate emotional expressions. Experiments show that our proposed approaches can generate appropriate emotion and yield significant improvements over the baseline methods in emotional conversation.

ABSTRACT: We introduce a new task named Story Ending Generation (SEG), which aims at generating a coherent story ending from a sequence of story plot. We propose a framework consisting of a Generator and a Reward Manager for this task. The Generator follows the pointer-generator network with coverage mechanism to deal with out-of-vocabulary (OOV) and repetitive words. Moreover, a mixed loss method is introduced to enable the Generator to produce story endings of high semantic relevance with story plots. In the Reward Manager, the reward is computed to fine-tune the Generator with policy-gradient reinforcement learning (PGRL). We conduct experiments on the recently-introduced ROCStories Corpus. We evaluate our model in both automatic evaluation and human evaluation. Experimental results show that our model exceeds the sequence-to-sequence baseline model by 15.75% and 13.57% in terms of CIDEr and consistency score respectively.

ABSTRACT: Previous research on dialogue systems generally focuses on the conversation between two participants. Yet, group conversations which involve more than two participants within one session bring up a more complicated situation. The scenario is real such as meetings or online chatting rooms. Learning to converse in groups is challenging due to different interaction patterns among users when they exchange messages with each other. Group conversations are structure-aware while the structure results from different interactions among different users. In this paper, we have an interesting observation that fewer contexts can lead to better performance by tackling the structure of group conversations. We conduct experiments on the public Ubuntu Multi-Party Conversation Corpus and the experiment results demonstrate that our model outperforms baselines.

ABSTRACT: Semantic ellipsis of the de structure makes it difficult for the machine to understand semantics automatically. To slove the classification of the de structure containing the usage of semantic ellipsis, a hybrid neural network is built. Firstly, the network uses a bidirectional LSTM(Long Short-Term Memory) neural network to learn more syntactic and semantic information of the de structure. Then, the network employs Max-Pooling layer or GRU(Gated Recurrent Unit) based multiple attention layers to capture features of ellipsis of the de structure by which the network can recognize the de structure containing the usage of semantic ellipsis. Experiments on CTB8.0 corpus show that the proposed approach can achieve efficient and accurate results.

ABSTRACT: Document similarity calculation is the basic work of legal case retrieval technology. At present, there are many document similarity calculation methods based on Siamese network. In view of the existing research, the entire document is regarded as the input sequence of model that may lead to sparse data. This paper uses hierarchical attention mechanism to improve the document representation in the Siamese network; For the Siamese network computing model based on hierarchical attention mechanism may ignore the important sentence in the document when inputting, we further propose a two-step document similarity calculation method that introduces the compression of document content. The experimental results show that the proposed method is obviously superior to the Siamese network computing model based on the Long Short-Term Memory.

ABSTRACT: Identifying event instance in texts plays a critical role in the field of Information Extraction (IE). The currently proposed methods that employ neural networks have successfully solve the problem to some extent, by encoding a series of linguistic features, such as lexicon, part-of-speech and entity. However, so far, the entity relation hasn’t yet been taken into consideration. In this paper, we propose a novel event extraction method to exploit relation information for event detection (ED), due to the potential relevance between entity relation and event type. Methodologically, we combine relation and those widely used features in an attention-based network with Bidirectional Long Short-term Memory (Bi-LSTM) units. In particular, we systematically investigate the effect of relation representation between entities. In addition, we also use different attention strategies in the model. Experimental results show that our approach outperforms other state-of-the-art methods.

ABSTRACT: Knowledge graph completion aims to find new true links between entities. In this paper, we consider the approach to embedding a knowledge graph into a continuous vector space. Embedding methods, such as TransE, TransR, and ProjE, are proposed in recent years and have achieved promising predictive performance. We discuss that a lot of substructures related with different relation properties in knowledge graph should be considered during embedding. We list 8 kinds of substructures and find that none of the existing embedding methods could encode all the substructures properly. Considering the structure diversity, we propose that a knowledge graph embedding method should have diverse representations for entities in different relation contexts and different entity positions. And we propose a new embedding method ProjR which combines TransR and ProjE together to achieve diverse representations by defining a unique combination operator for each relation. In ProjR, the input head entity-relation pairs with different relations will go through a different combination process. We conduct experiments with link prediction task on benchmark datasets for knowledge graph completion and the experiment results show that, with diverse representations, ProjR performs better compared with TransR and ProjE. We also analyze the performance of ProjR in the 8 different substructures listed in this paper and the results show that ProjR achieves better performance in most of the substructures.

ABSTRACT: In Recent years, medical text mining has been an active research field because of its significant application potential, and information extraction (IE) is an essential step in it. This paper focuses on the medical IE, whose aim is to extract the pivotal contents from the medical texts such as drugs, treatments and so on. In existing works, introducing side information into neural network based Conditional Random Fields (CRFs) models has been verified to be effective and widely used in IE. However, they always neglect the traditional attributes of data, which are important for the IE performance, such as lexical and morphological information. Therefore, starting from the raw data, a novel attribute embedding based MC-BLSTM-CRF model is proposed in this paper. We first exploit a bidirectional LSTM (BLSTM) layer to capture the context semantic information. Meanwhile, a multi-channel convolutional neural network (MC-CNN) layer is constructed to learn the relations between multiple attributes automatically and flexibly. And on top of these two layers, we introduce a CRF layer to predict the output labels. We evaluate our model on a Chinese medical dataset and obtain the state-of-the-art performance with 80.71% F1 score.

ABSTRACT: Recently, the seq2seq abstractive summarization models have achieved good results on the CNN/Daily Mail dataset. Still, how to improve abstractive methods with extractive methods is a good research direction, since extractive methods have their potentials of exploiting various efficient features for extracting important sentences in one text. In this paper, in order to improve the semantic relevance of abstractive summaries, we adopt the WordNet based sentence ranking algorithm to extract the sentences which are most semantically to one text. Then, we design a dual attentional seq2seq framework to generate summaries with consideration of the extracted information. At the same time, we combine pointer-generator and coverage mechanisms to solve the problems of out-of-vocabulary (OOV) words and duplicate words which exist in the abstractive models. Experiments on the CNN/Daily Mail dataset show that our models achieve competitive performance with the state-of-the-art ROUGE scores. Human evaluations also show that the summaries generated by our models have high semantic relevance to the original text.

ABSTRACT: Hypernym relation classification is one of the important tasks in natural language processing. Given two words, hypernym relation classification aims to judge whether the two words have hypernym-hyponym relation. Currently, the most mainstream methods for hypernym relation classification are the path-based method and the distributional method. The path-based method conducts classification based on the pattern features of the word pair in sentences. Due to the sparsity of pattern features, the path-based method suffers from low recall. The distributional method conducts classification by directly using word embeddings trained on large-scale corpus, but fails to make full use of the contextual information of the word pair in sentences. This paper proposes a word pattern based approach, which can effectively alleviate the sparsity problem suffered by the traditional path-based method. Furthermore, this paper makes an effective combination of the path-based method and the distributional method via word pattern embedding. To demonstrate the effectiveness of our approach, we have manually annotated a Chinese hypernym dataset containing 12,000 word pairs. The experimental results show that our proposed word pattern embedding approach is effective and can achieve an F1 score of 95.36.

ABSTRACT: Nested named entities contain rich entities and semantic relations between them, so nested named entity relation extraction can expand factual knowledge in the knowledge base effectively. However, due to the lack of corresponding benchmark corpora, no further research can be conducted in this direction. Therefore, this paper proposes to manually annotate a Chinese nested named entity relation corpus from an existing Chinese named entity recognition and further experiments with relation extraction between nested named entities via Support Vector Machines and Convolutional Neural Network models respectively. The experimental results show that the nested entity relation extraction performs excellently on the corpus with manually labeled entities, obtaining an F1 score of over 95%, while it fall short of expectations with automatically recognized entities.

ABSTRACT: Analysis of the character structure characteristics can lay an information foundation for the intelligent processing of square Hmong characters. Combined with the analysis of character structure characteristics, this paper presents a definition of the linearization of square Hmong characters, a definition of equivalence class division of the structure of square Hmong characters, and proposes a decision algorithm of structure equivalence class. According to the above algorithm, the structure of square Hmong characters is divided into eight equivalent classes. Analysis of the statistical properties, including the cumulative probability distribution, complexity, and information entropy of square Hmong characters appearing in practical documents, shows that, first, more than 90% of square Hmong characters appearing in practical documents are composed of two components, and more than 80% of these characters possess a left–right, top–bottom, or lower-left-enclosed structure, second, the number of mean components in a square Hmong character is slightly greater than 2, third, the information entropy of the structure of Hmong characters is within the interval (1.19, 2.16). Results reveal that square Hmong characters appearing frequently in practical documents follow the principle of simple structure orientation.

ABSTRACT: Off-colour humour is a category of humour which is considered by many to be in poor taste or overly vulgar. Most commonly, off-colour humour contains remarks on particular ethnic group or gender, violence, domestic abuse, acts concerned with sex, excessive swearing or profanity. Blue humour, black humour and insult humour are types of off-colour humour. Blue and black humour unlike insult humour are not outrightly insulting in nature but are often misclassified because of the presence of insults and harmful speech. As the primary contributions of this paper we provide an original data-set consisting of nearly 15,000 instances and a novel approach towards resolving the problem of separating black and blue humour from offensive humour which is essential so that free-speech on the internet is not curtailed. Our experiments show that deep learning methods outperforms other n-grams based approaches like SVM’s, Naive Bayes and Logistic Regression by a large margin.

ABSTRACT: Short-term prediction of stock market trend has potential application for personal investment without high-frequency-trading infrastructure. Existing studies on stock market trend prediction have introduced machine learning methods with handcrafted features. However, manual labor spent on handcrafting features is expensive. To reduce manual labor, we propose a novel recurrent convolutional neural network for predicting stock market trend. Our network can automatically capture useful information from news on stock market without any handcrafted features. In our network, we first introduce an embedding layer to automatically learn word embedding using financial news. We then use a convolutional layer to extract key information affecting stock market trend, and use a long-short term memory neural network to learn context-dependent relations in financial news for stock market trend prediction. Experimental results show that our model can achieve significant improvement by nearly 3% in terms of individual stock predictions, compared with the state-of-the-art baseline methods.

ABSTRACT: User profiling in social media plays an important role in different applications. Most of the existing approaches for user profiling are based on user-generated messages, which is not sufficient for inferring user attributes. With the continuous accumulation of data in social media, integrating multi-data sources has become the inexorable trend for precise user profiling. In this paper, we take advantage of text messages, user metadata, followee information and network representations. In order to integrate seamlessly multi-data sources, we propose a novel fusion model that effectively captures the complementarity and diversity of different sources. In addition, we address the problem of friendship-based network from previous studies and introduce celebrity ties which enrich the social network and boost the connectivity of different users. Experimental results show that our method outperforms several state-of-the-art methods on a real-world data set.

ABSTRACT: Abstract With the rapid development of movie industry, movie watching has become one of the most popular ways of people’s spiritual entertainments. The movie ranking prediction is a hot research domain for the guidance to the people’s viewing selections and the screening number of cinema for movies. In this paper, by combining learning to rank methods, we proposed a ranking prediction model by mining and analyzing the data from movie media website, which includes extracting and expanding features related to ranking prediction as well as dividing and aligning ranking labels etc. Experiment results show that our model effectively improved the performance of the movie ranking prediction task, which can benefit the cinemas to arrange the number of screenings properly. Besides that, the model can also provide high quality recommendations to movies for the fans. Hence, our proposed model is of certain practical value.

ABSTRACT: Word embedding has been used as a popular feature in various Natural Language Processing(NLP) tasks. To overcome the coverage problem of statistics, compositional model is proposed, which embeds basic units of a language, and compose structures of higher hierarchy, like idiom, phrase，and named entity. In that case, selecting the right level of basic-unit embedding to represent semantics of higher hierarchy unit is crucial. This paper investigates this problem by Chinese phrase representation task, in which language characters and words are viewed as basic units. We define a phrase representation evaluation tasks by utilizing Wikipedia. We propose four intuitionistic composing methods from basic embedding to higher level representation, and investigate the performance of the two basic units. Empirical results show that with all composing methods, word embedding out performs character embedding on both tasks, which indicates that word level is more suitable for composing semantic representation.

ABSTRACT: Word embedding is a distributed representation of words in a vector space. It involves a mathematical embedding from a space with one dimension per word to a continuous vector space with much lower dimension. It performs well on tasks including synonym and hyponym detection by grouping similar words. However, most existing word embeddings are insensitive to antonyms, since they are trained based on word distributions in a large amount of text data, where antonyms usually have similar contexts. To generate word embeddings that are capable of detecting antonyms, we firstly modify the objective function of Skip-Gram model, and then utilize the supervised synonym and antonym information in thesauri as well as the sentiment information of each word in SentiWordNet. We conduct evaluations on three relevant tasks, namely GRE antonym detection, word similarity, and semantic textual similarity. The experiment results show that our antonym-sensitive embedding outperforms common word embeddings in these tasks, demonstrating the efficacy of our methods.

ABSTRACT: Recently the deep learning techniques have achieved success in multi-label classification due to its automatic representation learning ability and the end-to-end learning framework. Existing deep neural networks in multi-label classification can be divided into two kinds: binary relevance neural network (BRNN) and threshold dependent neural network (TDNN). However, the former needs to train a set of isolate binary networks which ignore dependencies between labels and have heavy computational load, while the latter needs an additional threshold function mechanism to transform the multi-class probabilities to multi-label outputs. In this paper, we propose a joint binary neural network (JBNN), to address these shortcomings. In JBNN, the representation of the text is fed to a set of logistic functions instead of a softmax function, and the multiple binary classifications are carried out synchronously in one neural network framework. Moreover, the relations between labels are captured via training on a joint binary cross entropy (JBCE) loss. To better meet multi-label emotion classification, we further proposed to incorporate the prior label relations into the JBCE loss. The experimental results on the benchmark dataset show that our model performs significantly better than the state-of-the-art multi-label emotion classification methods, in both classification performance and computational efficiency.

ABSTRACT: Dependency parsing is an important NLP task. A popular approach for dependency parsing is structured perceptron. Still, graph-based dependency parsing has the time complexity of O(n^3), and it suffers from slow training. To deal with this problem, we propose a parallel algorithm called parallel perceptron. The parallel algorithm can make full use of a multi-core computer which saves a lot of training time. Based on experiments we observe that dependency parsing with parallel perceptron can achieve 8-fold faster training speed than traditional structured perceptron methods when using 10 threads, and with no loss at all in accuracy.

ABSTRACT: In recent years, Word Embedding has gradually become the standard input of deep learning natural language processing models. However, the method based on pre-training still has some defects in the stability and the quality of low-frequency words. This paper proposes a new word embedding method based on Hownet: first of all, based on the sememe independence assumption all sememes of Hownet are specified in an Euclidean Space’s standard orthogonal basis, which to initialize all sememe vectors; Secondly, utilizing the relationship between word and sememe defined in the Hownet, each word vector representation can be regarded as a subspace projection by related sememes; finally, we put forward a deep neural network model to learn word representations. The experimental results indicate that this word Embedding method based on Hownet has obtained comparable results in the two standard evaluation tasks including the word similarity computation and the word sense disambiguation.

ABSTRACT: The authors introduce a neural network library for Natural Language Processing named N3LDG. When using deep learning method to resolve NLP tasks, it’s often desirable to dynamically construct a computation graph, and then to organize executions into a batch. It’s difficult to construct computation graphs dynamically through libraries such as Theano and TensorFlow, while is possible through PyTorch, despite the extra difficulty of batching manually.N3LDGsupport constructing computation graphs dynamically, and organizing executions into batches automatically, which facilitates NLP researchers. Experiments show that N3LDG can efficiently build and execute computation graphs in CNN, bi-LSTM, and Tree-LSTM, where all models on the CPU perform faster than PyTorch. The convolutional neural network and tree LSTM models on the GPU perform faster than PyTorch.N3LDGis released at https://github.com/zhangmeishan/N3LDG, under Apache 2.0 license.

ABSTRACT: Almost all the state-of-the-art methods for Character-based Chinese dependency parsing ignore the complete dependency subtree information built during the parsing process, which is crucial for parsing the rest part of the sentence. In this paper, we introduce a novel neural network architecture to capture dependency subtree feature. We extend and improve recent works in neural joint model for Chinese word segmentation, POS tagging and dependency parsing, and adopt bidirectional LSTM to learn n-gram feature representation and context information. The neural network and bidirectional LSTMs are trained jointly with the parser objective, resulting in very effective feature extractors for parsing. Finally, we conduct experiments on Penn Chinese Treebank 5, and demonstrate the effectiveness of the approach by applying it to a greedy transition-based parser. The results show that our model outperforms the state-of-the-art neural joint models in Chinese word segmentation, POS tagging and dependency parsing.

ABSTRACT: This paper focuses on automatic question generation (QG) that transforms a narrative sentence into an interrogative sentence. Recently, neural networks have been used in this task due to its extraordinary ability of semantics encoding and decoding. We propose an approach which incorporates semantics of the possible question type. We utilize the Convolutional Neural Network (CNN) for predicting question type of the answer phrases in the narrative sentence. In order to incorporate the question type semantics into the generating process, we classify the question type which the answer phrases refer to. In addition, we use Bidirectional Long Short Term Memory (Bi-LSTM) to construct the question generating model. The experiment results show that our method outperforms the baseline system with the improvement of 1.7% on BLEU-4 score and beyond the state-of-the-art.

ABSTRACT: User intent identification and classification has become a vital topic of query understanding in human-computer dialogue applications. The identification of users’ intent is especially crucial for assisting system to understand users’ queries so as to classify the queries accurately to improve users’ satisfaction. Since the posted queries are usually short and lack of context, conventional methods heavily relied on query n-grams or other common features are not sufficient enough. This paper proposes a compact yet effective user intention classification method named as ST-UIC based on a constructed semantic tag repository. The method proposes to use a combination of four kinds of features including characters, non-key-noun part-of-speech tags, target words, and semantic tags. The experiments are based on a widely applied dataset provided by the First Evaluation of Chinese Human-Computer Dialogue Technology. The result shows that the method achieved a F1 score of 0.945, exceeding a list of baseline methods and demonstrating its effectiveness in user intent classification.

ABSTRACT: This paper tackles the task of event detection, which involves identifying and categorizing the events. Currently event detection remains a challenging task due to the difficulty at encoding the event semantics in complicate contexts. The core semantics of an event may derive from its trigger and arguments. However, most of previous studies failed to capture the argument semantics in event detection. To address this issue, this paper first provides a rule-based method to predict candidate arguments on the event types of possibilities, and then proposes a recurrent neural network model RNN-ARG with the attention mechanism for event detection to capture meaningful semantic regularities form these predicted candidate arguments. The experimental results on the ACE 2005 English corpus show that our approach achieves competitive results compared with previous work.

ABSTRACT: Event coreference resolution is a challenging NLP task due to this task needs to understand the semantics of events. Different with most previous studies used probability-based or graph-based models, this paper introduces a novel neural network, MDAN (Multiple Decomposable Attention Networks), to resolve document-level event coreference from different views, i.e., event mention, event arguments and trigger context. Moreover, it applies a document-level global inference mechanism to further resolve the coreference chains. The experimental results on two popular datasets ACE and TAC-KBP illustrate that our model outperforms the two state-of-the-art baselines.

ABSTRACT: Event-Event Relation Detection (RD2e) aims to detect the relations between a pair of news events, such as Causal relation between Criminal and Penal events. In general, RD2e is a challenging task due to the lack of explicit linguistic feature signaling the relations. We propose a cross-scenario inference method for RD2e. By utilizing conceptualized scenario expression and graph-based semantic distance perception, we retrieve semantically similar historical events from Gigaword. Based on explicit relations of historical events, we infer implicit relations of target events by means of transfer learning. Experiments on 10 relation types show that our method outperforms the supervised models.

ABSTRACT: Mining sub-event relations of major events is an important research problem, which is useful for building event taxonomy, event knowledge base construction, and natural language understanding. To advance the study of this problem, this paper presents a novel dataset called SeRI (Sub-event Relation Inference). SeRI includes 3,917 event articles from English Wikipedia and the annotations of their sub-events. It can be used for training or evaluating a model that mines sub-event relation from encyclopedia-style texts. Based on this dataset, we formally define the task of sub-event relation inference from an encyclopedia, propose an experimental setting and evaluation metrics and evaluate some baseline approaches' performance on this dataset.

ABSTRACT: Deep neural networks have recently been shown to achieve highly competitive performance in many computer vision tasks due to their abilities of exploring in a much larger hypothesis space. However, since most deep architectures like stacked RNNs tend to suffer from the vanishing-gradient and overfitting problems, their effects are still understudied in many NLP tasks. Inspired by this, we propose a novel multilayer RNN model called densely connected bidirectional long short-term memory (DC-Bi-LSTM) in this paper, which essentially represents each layer by the concatenation of its hidden state and all preceding layers hidden states, followed by recursively passing each layers representation to all subsequent layers. We evaluate our proposed model on five benchmark datasets of sentence classification. DC-Bi-LSTM with depth up to 20 can be successfully trained and obtain significant improvements over the traditional Bi-LSTM with the same or even less parameters. Moreover, our model has promising performance compared with the state-of-the-art approaches.

ABSTRACT: Multi-task learning (MTL) models, which pool examples arisen out of several tasks, have achieved remarkable results in language processing. However, multi-task learning is not always effective when compared with the single-task methods in sequence tagging. One possible reason is that existing methods to multi-task sequence tagging often reply on lower layer parameter sharing to connect different tasks. The lack of interactions between different tasks results in limited performance improvement. In this paper, we propose a novel multi-task learning architecture which could iteratively utilize the prediction results of each task explicitly. We train our model for part-of-speech (POS) tagging, chunking and named entity recognition (NER) tasks simultaneously. Experimental results show that without any task-specific features, our model obtains the state-of-the-art performance on both chunking and NER.

ABSTRACT: Neural machine translation has become a benchmark method in machine translation. Many novel structures and methods have been proposed to improve the translation quality. However, it is difficult to train and turn parameters. In this paper, we focus on decoding techniques that boost translation performance by utilizing existing models. We address the problem from three aspects, parameter, word and sentence level, corresponding to checkpoint averaging, model ensembling and candidates reranking which all do not need to retrain the model. Experimental results have shown that the proposed decoding approaches can significantly improve the performance over baseline model.

ABSTRACT: Corpus is an essential resource for data driven natural language processing systems, especially for sentiment analysis. In recent years, people increasingly use emoticons on social media to express their emotions, attitudes or preferences. We believe that emoticons are a non-negligible feature of sentiment analysis tasks. However, few existing works focused on sentiment analysis with emoticons. And there are few related corpora with emoticons. In this paper, we create a large scale Chinese Emoticon Sentiment Corpus of Movies (CESCM). Different to other corpora, there are a wide variety of emoticons in this corpus. In addition, we did some baseline sentiment analysis work on CESCM. Experimental results show that emoticons do play an important role in sentiment analysis. Our goal is to make the corpus widely available, and we believe that it will offer great support to sentiment analysis research and emoticon research.

ABSTRACT: Affective analysis has received growing attention from both research community and industry. However, previous works either cannot express the complex and compound states of human's feelings or rely heavily on manual intervention. In this paper, by adopting Plutchik's wheel of emotions, we propose a low cost construction method that utilizes word embeddings and high-quality small seed-sets of affective words to generate multi-dimensional affective vector automatically. And a large-scale affective lexicon is constructed as a verification, which could map each word to a vector in the affective space. Meanwhile, the construction procedure uses little supervision or manual intervention, and could learn affective knowledge from huge amount of raw corpus automatically. Experimental results on affective classification task and contextual polarity disambiguation task demonstrate that the proposed affective lexicon outperforms other state-of-the-art affective lexicons.

ABSTRACT: We propose using convolution neural network (CNN) with active learning for information extraction of enterprise announcements. The training process of supervised deep learning model usually requires a large amount of training data with high-quality reference samples. Human production of such samples is tedious, and since inter-labeler agreement is low, very unreliable. Active learning helps assuage this problem by automatically selecting a small amount of unlabeled samples for humans to hand correct. Active learning chooses a selective set of samples to be labeled. Then the CNN is trained on the labeled data iteratively, until the expected experimental effect is achieved. We propose three sample selection methods based on certainty criterion. We also establish an enterprise announcements dataset for experiments, which contains 10410 samples totally. Our experiment results show that the amount of labeled data needed for a given extraction accuracy can be reduced by more than 45.79% compared to that without active learning.

ABSTRACT: The correct definition and recognition of sentences is the basis of NLP. For the characteristics of Chinese text structure, the theory of NT clause was proposed from the perspective of micro topics. Based on this theory, this paper proposes a novel method for construction NT clause. Firstly, this paper proposes a neural network model based on Attention and LSTM (Attention-LSTM), which can identify the location of the missing Naming, and uses manually annotated corpus to train the Attention-LSTM. Secondly, in the process of constructing NT clause, the trained Attention-LSTM is used to identify the location of the missing Naming. Then the NT clause can be constructed. The accuracy of the experimental result is 81.74%（+4.5%）. This paper can provide support for the task of text understanding, such as Machine Translation, Information Extraction, Man-machine Dialogue.

ABSTRACT: Complaint orders in mobile customer service are the records of complaint description, which professional knowledge and information on customer’s complaint intention are kept. Complaint orders classification is important and necessary to be established and completed for further mining, analysis and improving the quality of customer service. Constructed corpus is the basis of research. The lack of complaint orders classification corpus (COCC) in mobile customer service has limited the research of complaint orders classification. This paper first employs K-means algorithm and professional knowledge to determine complaint orders classification labels. Then we craft the annotation rules for complaint orders, and then construct complaint orders classification corpus. The corpus consists of 130044 complaint orders annotated. Finally, we statistically analyze the corpus constructed, and the agreement of each question class reaches over 91%. It indicates that the corpus constructed could provide a great support for complaint orders classification and specialized analysis.

ABSTRACT: The graph models are an important method in automatic text summarization. However, there will be problems of vector sparseness and information redundancy in text map to graph. In this paper, we propose a graph clustering summarization algorithm based on network representation learning. The sentences graph was construed by TF-IDF, and controlled the number of edges by a threshold. The Node2Vec is used to embedding the graph, and the sentences were clustered by k-means. Finally, the Modularity is used to control the number of clusters, and generating a brief summary of the document. The experiments on the MultiLing 2013 show the proposed algorithm improves the F-Score in ROUGE-1 and ROUGE-2.

ABSTRACT: Sentiment classification is an important task in the community of Nature Language Processing. This task aims to determine the sentiment category towards a piece of text. One challenging problem of this task is that it is difficult to obtain a large number of labeled samples. Therefore, a large number of studies are focused on semi-supervised learning, i.e., learning information from unlabeled samples. However, one disadvantage of the previous methods is that the unlabeled samples and the labeled samples are studied in different models, and there is no interaction between them. Based on this, this paper tackles the problem by proposing a semi-supervised sentiment classification based on auxiliary task learning, namely Aux-LSTM, which is used to assist learning the sentiment classification task with a small amount of human-annotated samples by training auto-annotated samples. Specifically, the two tasks are allowed to share the auxiliary LSTM layer, and the auxiliary expression obtained by the auxiliary LSTM layer is used to assist the main task. Empirical studies demonstrate that the proposed method can effectively improve the experimental performance.

ABSTRACT: Abstractive summarization based on seq2seq model is a popular research topic today. And pre-trained word embedding is a common unsupervised method to improve deep learning model’s performance in NLP. However, during applying this method directly to the seq2seq model, we find it does not achieve the same good result as other fields because of an over training problem. In this paper, we propose a normalized encoder-decoder structure to address it, which can prevent the semantic structure of pre-trained word embedding from being destroyed during training. Moreover, we use a novel focal loss function to help our model focus on those examples with low score for getting better performance. We conduct the experiments on NLPCC2018 share task 3: single document summary. Result showed that these two mechanisms are extremely useful, helping our model achieve state-of-the-art ROUGE scores and get the first place in this task from the current rankings.

ABSTRACT: In this paper, we report technique details of our approach for the NLPCC 2018 shared task knowledge-based question answering. Our system uses a word-based maximum matching method to find entity candidates. Then, we combine editor distance, character overlap andword2vec cosine similarity to rank SRO triples of each entity candidate. Finally, the object of the top 1 score SRO is selected as the answer of the question. The result of our system achieves 62.94% of answer exact matching on the test set.

ABSTRACT: Grammatical Error Correction (GEC) is an important task in natural language processing. In this paper, we introduce our system on NLPCC 2018 Shared Task 2 Grammatical Error Correction. The task is to detect and correct grammatical errors that occurred in Chinese essays written by non-native speakers of Mandarin Chinese. Our system is mainly based on the convolutional sequence-to-sequence model. We regard GEC as a translation task from the language of “bad” Chinese to the language of “good” Chinese. We describe the building process of the model in details. On the test data of NLPCC 2018 Shared Task 2, our system achieves the best precision score, and the F0:5 score is 29.02. Our final results ranked third among the participants.

ABSTRACT: Emotion detection in code-switching texts aims to identify the emotion labels of text which contains more than one language. The difficulties of this task include problems in bridging the gap between languages and capturing crucial semantic information for classification. To address these issues, we propose an ensemble model with sentiment words translation to build a powerful system. Our system first constructs an English-Chinese sentiment dictionary to make a connection between two languages. Afterwards, we separately train several models include CNN, RCNN and Attention based LSTM model. Then combine their classification results to improve the performance. The experiment result shows that our method has a good effect and achieves the second place among nineteen systems.

ABSTRACT: User profiling and personalized recommendation plays an important role in many business applications such as precision marketing and targeting advertisement. Since user data is heterogeneous, leveraging the heterogeneous information for user profiling and personalized recommendation is still a challenge. In this paper, we propose effective methods to solve two subtasks working in user profiling and recommendation. Subtask one is to predict users’ tags, we treat this subtask as a binary classification task, we combine users’ profile vector and social Large-scale Information Network Embedding (LINE) vector as user features, and use tag information as tag features, then apply a deep learning approach to predict which tags are related to a user. Subtask two is to predict the users a user would like to follow in the future. We adopt social based collaborative filtering (CF) to solve this task. Our results achieve second place in both subtasks.

ABSTRACT: This paper presents the overview of the shared task, emotion detection in code-switching text, in NLPCC 2018. The submitted systems are expected to automatically determine the emotions in the Chinese-English code switching text. Different from monolingual text, code-switching text contains more than one language, and the emotion can be expressed by either monolingual or bilingual form. Hence, the challenges are: how to integrate both monolingual and bilingual forms to detect emotion, and how to bridge the gap to between two languages. Our shared task has 19 team participants. The highest F-score was 0.515. In this paper, we introduce the task, the corpus, the participating teams, and the evaluation results.

ABSTRACT: In this paper, we give an overview for the shared task at the CCF Conference on Natural Language Processing & Chinese Computing (NLPCC 2018): Automatic Tagging of Zhihu Questions. The dataset is collected from the Chinese question-answering web site Zhihu, which consists 25551 tags and 721608 training samples in this shared task. This is a multi-label text classification task, and each question can have as much as five relevant tags. The dataset can be assessed at http://tcci.ccf.org.cn/conference/2018/taskdata.php

ABSTRACT: In this paper, we present an overview of the Grammatical Error Correction task in the NLPCC 2018 shared tasks. We give detailed descriptions of the task definition and the data for training as well as evaluation. We also summarize the approaches investigated by the participants of this task. Such approaches demonstrate the state-of-the-art of Grammatical Error Correction for Mandarin Chinese. The data set and evaluation tool used by this task is available at https://github.com/zhaoyyoo/NLPCC2018_GEC.

ABSTRACT: In this paper, we give an overview of multi-turn human computer conversations at NLPCC 2018 shared task. This task consists of two sub-tasks: conversation generation and retrieval with given context. Data-sets for both training and testing are collected from Weibo, where there are 5 million conversation sessions for training and 40,000 non-overlapping conversation sessions for evaluating. Details of the shared task, evaluation metric, and submitted models will be given successively.

ABSTRACT: We give the overview of the open domain QA shared task in the NLPCC 2018. In this year, we release three sub-tasks including Chinese knowledge-based question answering (KBQA) task, Chinese knowledge-based question generation (KBQG) task, and English knowledge-based question understanding (KBQU) task. The evaluation results of final submissions from participating teams will be presented in the experimental part.

ABSTRACT: In this report, we give an overview of the shared task about single document summarization at the seventh CCF Conference on Natural Language Processing and Chinese Computing (NLPCC 2018). Short summaries for articles are consumed frequently on mobile news apps. Because of the limited display space on the mobile phone screen, it is required to create concise text for the main idea of an article. This task aims at promoting technology development for single document summarization. We describe the task, the corpus, the participating teams and their results.

ABSTRACT: In this paper, we give the overview of the social media user modeling shared task in the NLPCC 2018. We first review the background of social media user modeling, and then describe two social media user modeling tasks in this year’s NLPCC, including the construction of the benchmark datasets and the evaluation metrics. The evaluation results of submissions from participating teams are presented in the experimental part.

ABSTRACT: This paper presents the overview for the shared task at the 7th CCF Conference on Natural Language Processing & Chinese Computing (NLPCC2018): Spoken Language Understanding (SLU) in Task-oriented Dialog Systems. SLU usually consists of two parts, namely intent identification and slot filling. The shared task made publicly available a Chinese dataset of over 5.8Ksessions, which is a sample of the real query log from a commercial task oriented dialog system and includes 26K utterances. The contexts within a session are taken into consideration when a query within the session was annotated. To help participating systems correct ASR errors of slot values, this task also provides a dictionary of values for each enumerable type of slot. 16 teams entered the task and submitted a total of 40 SLU results. In this paper, we will review the task, the corpus, and the evaluation results.

ABSTRACT: Multi-label topic classification aims to assign one or more relevant topic labels to a text. This paper presents the WiseTag system, which performs multi-label topic classification based on an ensemble of four single models, namely a KNN-based model, an Information Gain based model, a Keyword Matching-based model and a Deep Learning based model. These single models are carefully designed so that they are diverse enough to improve the performance of the ensemble model. In the NLPCC 2018 shared task 6 “Automatic Tagging of Zhihu Questions”,the proposed WiseTag system achieves an F1 score of 0.4863 on the test set, and ranks no. 4 among all the teams.

ABSTRACT: Traditional reading comprehension dataset, such as SQuAD, contains only one document for a given question, and the answer of this question could be found in one certain paragraph.However, in realworld reading comprehension, an answer to a question is usually extracted from multiple paragraphs in multiple documents. Thus it is vital important to first select some paragraphs that related to the question, then reading comprehension is applied to generate answer from those related paragraphs.In this paper, we investigate various paragraph selection techniques based on bidirectional attention flow and convolutional neural network. We evaluate our approaches on DuReader, a large scale Chinese dataset for reading comprehension with multiple documents. Experiments show that our proposed method improves reading comprehension performance compared to a baseline based on maximum question coverage.

ABSTRACT: Machine Reading Comprehension (MRC) is a challenging task in the field of Natural Language Processing (NLP) and Artificial Intelligence (AI). 2018 NLP Challenge on Machine Reading Comprehension (MRC2018) aims to advance MRC technologies and applications. The challenge provides a large-scale, open-domain, application-oriented Chinese MRC dataset; releases open sourced baseline systems and adopts improved evaluation metrics. Over a thousand teams registered for this challenge and the overall performance of the participant systems have been greatly promoted. This talk presents an overall introduction to MRC2018, and gives a detailed description of the evaluation task settings, evaluation organization, evaluation results and corresponding result analysis.