Analysis of the Effectiveness of CNN-LSTM Models Incorporating Bert and Attention Mechanisms in Sentiment Analysis of Data Reviews

DOI

10.2991/978-94-6463-238-5_106How to use a DOI?

Keywords

Bert; convolutional neural networks; long- and short-term memory neural networks; attentional mechanisms

Abstract

This paper proposes a CNN-LSTM model based on Bert and attention mechanism, since current models cannot deal well with long-term dependencies in natural language. Firstly, the Bert-encoded text vector is fed into the CNN-LSTM model, and secondly, the output of the CNN-LSTM model is fed into the Attention-Based layer, which extracts the most relevant information from the input, and the important features are extracted by weighting the vector. The results show that compared with BiLSTM-ATT, Hierarchical Attention Network (HAN), Convolutional Neural Network (ABCNN), and Attention-Based models, the proposed model has significantly improved in accuracy, F1 score, and macro-averaged F1 metrics. The proposed model has significantly improved in accuracy, F1 score, and macro-average F1 metrics.

Copyright

© 2024 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Lujuan Deng
AU  - Tiantian Yin
AU  - Zuhe Li
AU  - Qingxia Ge
PY  - 2023
DA  - 2023/09/26
TI  - Analysis of the Effectiveness of CNN-LSTM Models Incorporating Bert and Attention Mechanisms in Sentiment Analysis of Data Reviews
BT  - Proceedings of the 2023 4th International Conference on Big Data and Informatization Education (ICBDIE 2023)
PB  - Atlantis Press
SP  - 821
EP  - 829
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-238-5_106
DO  - 10.2991/978-94-6463-238-5_106
ID  - Deng2023
ER  -