Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification
- 10.2991/ijcis.d.190710.001How to use a DOI?
- Natural language processing; Neural network; Gated recurrent units; Text classification
Recurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results based on RNN. However, most of these models overlook the locations of the keywords in a sentence and the semantic connections in different directions. As a consequence, these methods do not make full use of the available information. Considering that different words in a sequence usually have different importance, in this paper, we propose bidirectional gated recurrent units (BGRUs) which integrates a novel attention pooling mechanism with max-pooling operation to force the model to pay attention to the keywords in a sentence and maintain the most meaningful information of the text automatically. The presented model allows to encode longer sequences. Thus, it not only prevents important information from being discarded but also can be used to filter noises. To avoid full exposure of content without any control, we add an output gate to the GRU, which is named as text unit. The proposed model was evaluated on multiple tasks, including sentimental classification, movie review data, and a subjective classification dataset. Experimental results show that our model can achieve excellent performance on these tasks.
- © 2019 The Authors. Published by Atlantis Press SARL.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
Cite this article
TY - JOUR AU - Dejun Zhang AU - Mingbo Hong AU - Lu Zou AU - Fei Han AU - Fazhi He AU - Zhigang Tu AU - Yafeng Ren PY - 2019 DA - 2019/07/12 TI - Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification JO - International Journal of Computational Intelligence Systems SP - 723 EP - 732 VL - 12 IS - 2 SN - 1875-6883 UR - https://doi.org/10.2991/ijcis.d.190710.001 DO - 10.2991/ijcis.d.190710.001 ID - Zhang2019 ER -