International Journal of Computational Intelligence Systems

<Previous Article In Issue
Next Article In Issue>
Volume 12, Issue 2, 2019, Pages 723 - 732

Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification

Authors
Dejun Zhang1, *, Mingbo Hong2, Lu Zou2, Fei Han2, Fazhi He3, Zhigang Tu4, Yafeng Ren5
1Faculty of Information Engineering, China University of Geosciences, Wuhan, Hubei 430074, China
2College of Information and Engineering, Sichuan Agricultural University, Yaan, Sichuan 625014, China
3School of Computer, Wuhan University, Wuhan, Hubei 430072, China
4School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 637553, Singapore
5Collaborative Innovation Center for Language Research & Service, Guangdong University of Foreign Studies, Guangzhou, Guangdong 510420, China
*Corresponding author. Email: zhangdejun@cug.edu.cn
Corresponding Author
Dejun Zhang
Received 8 May 2018, Accepted 27 June 2019, Available Online 12 July 2019.
DOI
10.2991/ijcis.d.190710.001How to use a DOI?
Keywords
Natural language processing; Neural network; Gated recurrent units; Text classification
Abstract

Recurrent neural network (RNN) is one of the most popular architectures for addressing variable sequence text, and it shows outstanding results in many natural language processing (NLP) tasks and remarkable performance in capturing long-term dependencies. Many models have achieved excellent results based on RNN. However, most of these models overlook the locations of the keywords in a sentence and the semantic connections in different directions. As a consequence, these methods do not make full use of the available information. Considering that different words in a sequence usually have different importance, in this paper, we propose bidirectional gated recurrent units (BGRUs) which integrates a novel attention pooling mechanism with max-pooling operation to force the model to pay attention to the keywords in a sentence and maintain the most meaningful information of the text automatically. The presented model allows to encode longer sequences. Thus, it not only prevents important information from being discarded but also can be used to filter noises. To avoid full exposure of content without any control, we add an output gate to the GRU, which is named as text unit. The proposed model was evaluated on multiple tasks, including sentimental classification, movie review data, and a subjective classification dataset. Experimental results show that our model can achieve excellent performance on these tasks.

Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

<Previous Article In Issue
Next Article In Issue>
Journal
International Journal of Computational Intelligence Systems
Volume-Issue
12 - 2
Pages
723 - 732
Publication Date
2019/07/12
ISSN (Online)
1875-6883
ISSN (Print)
1875-6891
DOI
10.2991/ijcis.d.190710.001How to use a DOI?
Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

risenwbib
TY  - JOUR
AU  - Dejun Zhang
AU  - Mingbo Hong
AU  - Lu Zou
AU  - Fei Han
AU  - Fazhi He
AU  - Zhigang Tu
AU  - Yafeng Ren
PY  - 2019
DA  - 2019/07/12
TI  - Attention Pooling-Based Bidirectional Gated Recurrent Units Model for Sentimental Classification
JO  - International Journal of Computational Intelligence Systems
SP  - 723
EP  - 732
VL  - 12
IS  - 2
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.d.190710.001
DO  - 10.2991/ijcis.d.190710.001
ID  - Zhang2019
ER  -