№2, 2024
COMPARISON OF DEEP LEARNING TECHNIQUES FOR TEXTUAL SENTIMENT ANALYSIS
Examining the opinions of the people might provide us with valuable knowledge. Sentiment analysis is a method for analyzing textual data that helps find subjective information, such as opinions and feelings that individuals or groups have expressed. It improves our understanding of human language using deep learning and natural language techniques. Several deep learning models, including RNNs, LSTMs, GRUs, and their bidirectional variants, are compared in this work. Three publicly available datasets - the imdb_reviews, Twitter Sentiment Dataset, and Emotions dataset were used in the investigation. Accuracy performance is evaluated for six deep learning models. According to experimental studies, bidirectional structures outperform their unidirectional counterparts in most cases. Across several datasets, the bidirectional models continuously produced the best accuracy (pp.32-40).
Ahmad, A. M., Ismail, S., & Samaon, D. F. (2004). Recurrent neural network with backpropagation through time for speech recognition. In IEEE International Symposium on Communications and Information Technology (ISCIT), Sapporo, Japan, October 2004 (pp. 98-102). https://doi.org/10.1109/ISCIT.2004.1412458
Alguliyev, R. M., Aliguliyev, R. M., & Abdullayeva, F. J. (2019). Deep learning method for prediction of DDoS attacks on social media. Advances in Data Science and Adaptive Analysis, 11(01n02), 1950002. https://doi.org/10.1142/S2424922X19500025
Alguliyev, R. M., Aliguliyev, R. M., & Abdullayeva, F. J. (2019). The improved LSTM and CNN models for DDoS attacks prediction in social media. International Journal of Cyber Warfare and Terrorism (IJCWT), 9(1), 1-18. https://doi.org/10.4018/IJCWT.2019010101
Alguliyev, R. M., Aliguliyev, R. M., & Niftaliyeva, G. Y. (2019). Extracting social networks from e-government by sentiment analysis of users' comments. Electronic Government, an International Journal, 15(1), 91-106. https://doi.org/10.1504/EG.2019.096576
Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2), 157-166. https://doi.org/10.1109/72.279181
Berglund, M., Raiko, T., Honkala, M., Kärkkäinen, L., Vetek, A., & Karhunen, J. T. (2015). Bidirectional recurrent neural networks as generative models. Advances in neural information processing systems. arXiv preprint arXiv: 1504.01575.
Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv: 1409.1259.
Connor, R., Dearle, A., Claydon, B., & Vadicamo, L. (2024). Correlations of cross-entropy loss in machine learning. Entropy, 26(6), 491. https://doi.org/10.3390/e26060491
Dadoun, A., & Troncy, R. (2020). Many-to-one recurrent neural network for session-based recommendation. arXiv preprint arXiv: 2008.11136.
Dey, R., & Salem, F. M. (2017). Gate-variants of gated recurrent unit (GRU) neural networks. In 60th IEEE International Midwest Symposium on Circuits and Systems (MWSCAS), Boston, USA, August 2017 (pp. 1597-1600). https://doi.org/10.1109/MWSCAS.2017.8053243
Gaafar, A. S., Dahr, J. M., & Hamoud, A. K. (2022). Comparative analysis of performance of deep learning classification approach based on LSTM-RNN for textual and image datasets. Informatica, 46(5), 21-28. https://doi.org/10.31449/inf.v46i5.3872
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural computation, 12(10), 2451-2471. https://doi.org/10.1049/cp:19991218
Graves, A., Mohamed, A. R., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In IEEE international conference on acoustics, speech and signal processing, Vancouver, Canada, May 2013 (pp. 6645-6649). https://doi.org/10.1109/ICASSP.2013.6638947
Hassan, A., & Mahmood, A. (2017, April). Deep learning approach for sentiment analysis of short texts. In 3rd international conference on control, automation and robotics (ICCAR), Nagoya, Japan, April 2017 (pp. 705-710). https://doi.org/10.1109/ICCAR.2017.7942788
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hussein, S. (2021). Twitter Sentiments Dataset. Mendeley Data. https://doi.org/10.17632/z9zw7nt5h2.1
Imrana, Y., Xiang, Y., Ali, L., & Abdul-Rauf, Z. (2021). A bidirectional LSTM deep learning approach for intrusion detection. Expert Systems with Applications, 185, 115524. https://doi.org/10.1016/j.eswa.2021.115524
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539
Li, S., Li, W., Cook, C., Zhu, C., & Gao, Y. (2018). Independently recurrent neural network (indRNN): Building a longer and deeper RNN. In IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, June 2018 (pp. 5457-5466). https://doi.org/10.1109/CVPR.2018.00572
Liu, X., You, J., Wu, Y., Li, T., Li, L., Zhang, Z., & Ge, J. (2020). Attention-based bidirectional GRU networks for efficient HTTPS traffic classification. Information Sciences, 541, 297-315. https://doi.org/10.1016/j.ins.2020.05.035
Lynn, H. M., Pan, S. B., & Kim, P. (2019). A deep bidirectional GRU network model for biometric electrocardiogram classification based on recurrent neural networks. IEEE Access, 7, 145395-145405.
https://doi.org/10.1109/ACCESS.2019.2939947
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011). Learning word vectors for sentiment analysis. In 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, USA, June 2011 (pp. 142-150).
Mousa, A., & Schuller, B. (2017). Contextual bidirectional long short-term memory recurrent neural network language models: A generative approach to sentiment analysis. In 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, April 2017 (pp. 1023–1032).
Pandya, D., & Thakkar, A. (2024). Sentiment Analysis of Self Driving Car Dataset: A comparative study of Deep Learning approaches. Procedia Computer Science, 235, 12-21. https://doi.org/10.1016/j.procs.2024.04.002
Park, J., Yi, D., & Ji, S. (2020). Analysis of recurrent neural network and predictions. Symmetry, 12(4), 615. https://doi.org/10.3390/sym12040615
Saravia, E., Liu, H. C. T., Huang, Y. H., Wu, J., & Chen, Y. S. (2018). CARER: Contextualized affect representations for emotion recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October-November 2018 (pp. 3687-3697). https://doi.org/10.18653/v1/D18-1404
Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673-2681. https://doi.org/10.1109/78.650093
Shiri, F. M., Perumal, T., Mustapha, N., & Mohamed, R. (2023a). A comprehensive overview and comparative analysis on deep learning models: CNN, RNN, LSTM, GRU. arXiv preprint arXiv:2305.17473.
Shiri, F., Perumal, T., Mustapha, N., Mohamed, R., Ahmadon, M. A. B., & Yamaguchi, S. (2023b). A Survey on Multi-Resident Activity Recognition in Smart Environments, arXiv preprint arXiv: 2304.12304.
Smagulova, K., & James, A. P. (2019). A survey on LSTM memristive neural network architectures and applications. The European Physical Journal Special Topics, 228(10), 2313-2324. https://doi.org/10.1140/epjst/e2019-900046-x
Tang, D., Qin, B., & Liu, T. (2015). Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, September 2015 (pp. 1422-1432).
Wankhade, M., Rao, A. C. S., & Kulkarni, C. (2022). A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55(7), 5731-5780. https://doi.org/10.1007/s10462-022-10144-1
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017). Comparative study of CNN and RNN for natural language processing. arXiv preprint arXiv:1702.01923.
