Sentiment Analysis toward the Use of MySAPK BKN Application in Google Play Store
Abstract
In realizing the Electronic-based Government System (Sistem Pemerintahan Berbasis Elektronik - SPBE) policy, the National Civil Service Agency (Badan Kepegawaian Nasional - BKN), as the fostering government agency for the State Civil Aparatus (Aparatur Sipil Negara - ASN), needs to carry out an accurate, up-to-date, and integrated data management through an Android-based application named MySAPK. Over time, users encounter various troubles operating this application and writing down their reviews on the Rating & Review feature in Google Play Store. From May 9, 2017, until October 18, 2021, reviews written by users amounted to 4,778. This paper conducted a sentiment analysis on MySAPK users’ reviews. Stages carried out included data collection, data labeling (annotation), data preprocessing, word feature extraction, classification modeling, modeling evaluation, sentiment analysis, and recommendation result preparation. The classification modeling of the sentiment using the naïve Bayes and support vector machine (SVM) resulted in an accuracy level of 92.47% and 94.14%, respectively. The sentiment measurement results showed that users who wrote reviews with positive sentiments amounted to 2,118 (44.3%), and negative sentiments amounted to 2,660 (55.7%). Factors that prompted users to leave positive sentiment reviews are the excellent application quality, providing benefits, making it easier to fill out and store ASN data, and gratitude comments for BKN. On the other hand, factors causing users to leave negative sentiments reviews include requesting to fix the application, having difficulty accessing the application, failing to fill out and update data, and encountering an error server. To address these issues, this paper suggests that BKN could increase supporting server capacity and update the most recent version to fix the bugs. The research result is expected to serve as a reference for BKN in evaluating and improving the quality of ASN services via the MySAPK app.
References
A. Arief and M.Y. Abbas, “Kajian Literatur (Systematic Literature Review): Kendala Penerapan Sistem Pemerintahan Berbasis Elektronik (SPBE),” PROtek: J. Ilm. Tek. Elektro, Vol. 8, No. 1, pp. 1–6, 2021.
“Sistem Pemerintahan Berbasis Elektronik,” Peraturan Presiden Republik Indonesia, No. 95, 2018.
“Satu Data Indonesia,” Peraturan Presiden Republik Indonesia, No. 39, 2019.
Badan Kepegawaian Negara, Buku Statistik ASN Juni 2021. Jakarta, Indonesia: Badan Kepegawaian Negara, 2021.
(2020) “Enam Tujuan Integrasi Data dalam Mewujudkan Satu Data ASN,” [Online], https://bkd.papua.go.id/2020/11/03/enam-tujuan-integrasi-data-dalam-mewujudkan-satu-data-asn, access date: 28-Nov-2021.
“Pemutakhiran Data Mandiri Aparatur Sipil Negara dan Pejabat Pimpinan Tinggi Non Aparatur Sipil Negara Secara Elektronik Tahun 2021,” Keputusan Kepala Kepegawaian Negara RI, No. 87, 2021.
(2021) “MySAPK BKN - Aplikasi di Google Play,” [Online], https://play.google.com/store/apps/details?id=com.nudi.bkn.sapk, access date: 18-Oct-2021.
(2021) “PDM ASN dan PPT Non ASN,” [Online], https://tiket-pdm.bkn.go.id/, access date: 28-Nov-2021.
A.N. Muhammad, S. Bukhori, and P. Pandunata, “Sentiment Analysis of Positive and Negative of YouTube Comments Using Naïve Bayes-Support Vector Machine (NBSVM) Classifier,” 2019 Int. Conf. Comput. Sci. Inf. Technol., Elect. Eng. (ICOMITEE), 2019, pp. 199–205.
I.C. Sari and Y. Ruldeviyani, “Sentiment Analysis of the Covid-19 Virus Infection in Indonesian Public Transportation on Twitter Data: A Case Study of Commuter Line Passengers,” 2020 Int. Workshop Big Data, Inf. Secur. (IWBIS), 2020, pp. 23–28.
S.M. Tauhid and Y. Ruldeviyani, “Sentiment Analysis of Indonesians Response to Influencer in Social Media,” 2020 7th Int. Conf. Inf. Technol., Comput., Elect. Eng. (ICITACEE), 2020, pp. 90–95.
A.S.H. Basari, B. Hussin, I.G.P. Ananta, and J. Zeniarja, “Opinion Mining of Movie Review Using Hybrid Method of Support Vector Machine and Particle Swarm Optimization,” Procedia Engineering, Vol. 53, pp. 453–462, 2013.
S. Fransiska and A.I. Gufroni, “Sentiment Analysis Provider by.U on Google Play Store Reviews with TF-IDF and Support Vector Machine (SVM) Method,” Sci. J. Inform., Vol. 7, No. 2, pp. 203–212, 2020.
B. Liu, Sentiment Analysis and Opinion Mining. London, England: Morgan & Claypool Publishers, 2012.
D. Westergaard, et al., “A Comprehensive and Quantitative Comparison of Text-Mining in 15 Million Full-Text Articles Versus Their Corresponding Abstracts,” PLOS Comput. Biol., Vol. 14, No. 2, pp. 1–16, 2018.
A.I. Dale, “Thomas Bayes’s Work on Infinite Series,” Hist. Math., Vol. 18, No. 4, pp. 312–327, 1991.
M.P. Geetha and D.K. Renuka, “Improving the Performance of Aspect based Sentiment Analysis Using Fine-Tuned Bert Base Uncased Model,” Int. J. Intell. Netw., Vol. 2, pp. 64–69, 2021.
V. Kevin, S. Que, A. Iriani, and H.D. Purnomo, “Analisis Sentimen Transportasi Online Menggunakan Support Vector Machine Berbasis Particle Swarm Optimization,” J. Nas. Tek. Elekt., Teknol. Inf., Vol. 9, No. 2, pp. 162–170, 2020.
W. Medhat, A. Hassan, and H. Korashy, “Sentiment Analysis Algorithms and Applications: A Survey,” Ain Shams Eng. J., Vol. 5, No. 4, pp. 1093–1113, 2014.
JoMingyu (2021) “google-play-scraper: Google Play Scraper for Python,” [Online], https://github.com/JoMingyu/google-play-scraper, access date: 14-Oct-2021.
Star Ratings Calculation Methodology, Press Ganey Associates, Inc., South Bend, USA, 2015.
D. Effrosynidis, S. Symeonidis, and A. Arampatzis, “A Comparison of Pre-processing Techniques for Twitter Sentiment Analysis,” in Res., Adv. Technol. Digit. Libraries. TPDL 2017. Lecture Notes in Computer Science, Vol. 10450, J. Kamps, et al., Eds., Cham, Switzerland: Springer, 2017, pp. 394–406.
F.Z. Tala, “A Study of Stemming Effects on Information Retrieval in Bahasa Indonesia,” Institute for Logic, Language, and Computation, Universiteit van Amsterdam, Amsterdam, Netherlands, Project Report, 2003.
H.A. Robbani (2018) “PySastrawi: Indonesian Stemmer,” [Online], https://github.com/har07/PySastrawi, access date: 01-Nov-2021.
M. Adriani, et al., “Stemming Indonesian,” ACM Trans. Asian Lang. Inf. Process., Vol. 6, No. 4, pp. 1–33, 2007.
R. Kusumawati, A. D’arofah, and P.A. Pramana, “Comparison Performance of Naive Bayes Classifier and Support Vector Machine Algorithm for Twitter’s Classification of Tokopedia Services,” J. Phys.: Conf. Ser., Vol. 1320, No. 1, pp. 1–10, 2019.
A.S. Neogi, K.A. Garg, R.K. Mishra, and Y.K. Dwivedi, “Sentiment Analysis and Classification of Indian Farmers’ Protest Using Twitter Data,” Int. J. Inf. Manage. Data Insights, Vol. 1, No. 2, pp. 1–11, 2021.
scikit-learn, “scikit-learn: Machine Learning in Python,” [Online], https://scikit-learn.org/stable, access date: 24-Nov-2021.
S. Al-Saqqa, G. Al-Naymat, and A. Awajan, “A Large-Scale Sentiment Data Classification for Online Reviews Under Apache Spark,” Procedia Comput. Sci., Vol. 141, pp. 183–189, 2018.
M. Birjali, M. Kasri, and A. Beni-Hssane, “A Comprehensive Survey on Sentiment Analysis: Approaches, Challenges and Trends,” Knowl-Based Syst., Vol. 226, pp. 1–26, 2021.
F. Heimerl, S. Lohmann, S. Lange, and T. Ertl, “Word Cloud Explorer: Text Analytics Based on Word Clouds,” 2014 47th Hawaii Int. Conf. Syst. Sci., 2014, pp. 1833–1842.
© Jurnal Nasional Teknik Elektro dan Teknologi Informasi, under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
1.png)
