Analisis Sentimen Review ChatGPT di Play Store menggunakan Support Vector Machine dan K-Nearest Neighbor
DOI:
https://doi.org/10.29408/edumatic.v8i1.24114Keywords:
chatgpt, sentiment, cross validation, svm, knnAbstract
The ChatGPT application for Android was launched on July 25, 2023, and the language model from OpenAI achieved a rating of 4.8 until early 2024. Despite the majority of positive reviews, user reports stating that ChatGPT provides inaccurate answers raise concerns about the reliability of this application. This research aims to compare the models of the Support Vector Machine (SVM) and K-Nearest Neighbor (KNN) algorithms in analyzing the sentiment of ChatGPT application reviews. Utilizing text mining methods to extract information from text, data was collected from Google Play Store reviews using data scraping techniques and analyzed with Support Vector Machine and K-Nearest Neighbor algorithms. Cross-validation with 5 folds and data split using 80% training and 20% testing data were applied to evaluate the performance of both algorithms. The sentiment classification results showed that the Support Vector Machine algorithm achieved an average accuracy of 80%, while K-Nearest Neighbor reached 71%. SVM excels due to its ability to overcome KNN's limitations regarding less relevant features that do not significantly contribute to predictions. The findings of this study are expected to help developers understand and respond to user feedback regarding the reliability of ChatGPT.
References
Alaei, A. R., Becken, S., & Stantic, B. (2017). Sentiment analysis in tourism: Capitalizing on big data. Journal of Travel Research, 58(2), 175–191. https://doi.org/10.1177/0047287517747753
Amrullah, A. Z., Anas, A. S., & Hidayat, M. A. J. (2020). Analisis Sentimen Movie Review Menggunakan Naive Bayes Classifier Dengan Seleksi Fitur Chi Square. Jurnal BITe, 2(1), 40–44. https://doi.org/10.30812/bite.v2i1.804
Atmajaya, D., Febrianti, A., & Darwis, H. (2023). Metode SVM dan Naive Bayes untuk Analisis Sentimen ChatGPT di Twitter. Indonesian Journal of Computer Science, 12(4). https://doi.org/10.33022/ijcs.v12i4.3341
Dowling, M., & Lucey, B. (2023). ChatGPT for (Finance) research: The Bananarama Conjecture. Finance Research Letters, 53, 103662. https://doi.org/10.1016/j.frl.2023.103662
Giovani, A. P., Ardiansyah, A., Haryanti, T., Kurniawati, L., & Gata, W. (2020). Analisis Sentimen Aplikasi Ruang Guru di Twitter Menggunakan Algoritma Klasifikasi. Jurnal Teknoinfo, 14(2), 115. https://doi.org/10.33365/jti.v14i2.679
Gravel, J., D’Amours-Gravel, M., & Osmanlliu, E. (2023). Learning to fake it: Limited responses and fabricated references provided by ChatGPT for medical questions. Mayo Clinic Proceedings: Digital Health, 1(3), 226–234. https://doi.org/10.1016/j.mcpdig.2023.05.004
Handayani, A. S., Soim, S., Agusdi, T. E., Rumiasih, & Nurdin, A., (2020). Klasifikasi Kualitas Udara Dengan Metode Support Vector Machine. Jurnal Informatika & Rekayasa Elektronika, 3(2), pp. 187-199.
Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: how ai-assisted conversational interfaces are revolutionizing the field. Big data and cognitive computing, 7(2), 62. https://doi.org/10.3390/bdcc7020062
Hidayatullah, H., Purwantoro, P., & Umaidah, Y. (2023). Penerapan Naive Bayes Dengan Oprimasi Information Gain Dan SMOTE Untuk Analisis Sentimen Pengguna Aplikasi ChatGPT. JATI (Jurnal Mahasiswa Teknik Informatika), 7(3), 1546–1553. https://doi.org/10.36040/jati.v7i3.6887
Krotov, V., Johnson, L., & Silva, L. (2020). Legality and ethics of web scraping. Communications of the Association for Information Systems, 47, 539–563. https://doi.org/10.17705/1cais.04724
Leippold, M. (2023). Thus spoke GPT-3: Interviewing a large-language model on climate finance. Finance Research Letters, 53(May 2023), 103617. https://doi.org/10.1016/j.frl.2022.103617
Lestari, S., Mupaat, M., & Erfina, A. (2022). Analisis Sentimen Masyarakat Indonesia terhadap Pemindahan Ibu Kota Negara Indonesia pada Twitter. JUSIFO (Jurnal Sistem Informasi), 8(1), 13–22. https://doi.org/10.19109/jusifo.v8i1.12116
Lailiyah, M., Sumpeno, S., & Purnama, I. K. E. (2017, August). Sentiment analysis of public complaints using lexical resources between Indonesian sentiment lexicon and Sentiwordnet. 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA). http://dx.doi.org/10.1109/isitia.2017.8124100
Liu, B., (2012). Sentiment Analysis and Opinion Mining. Morgan & Claypool Publishers.
Prasetyo, E. (2013). Data Mining : Konsep Dan Aplikasi Menggunakan Matlab. Yogyakarta: CV Andi Offset.
Rahayu, S., Mz, Y., Bororing, J. E., & Hadiyat, R. (2022). Implementasi Metode K-Nearest Neighbor (K-NN) untuk Analisis Sentimen Kepuasan Pengguna Aplikasi Teknologi Finansial FLIP. Edumatic: Jurnal Pendidikan Informatika, 6(1), 98–106. https://doi.org/10.29408/edumatic.v6i1.5433
Rifaldi, M., Ramadhan, Y., & Jaelani, I. (2023). Analisis Sentimen Terhadap Aplikasi Chatgpt Pada Twitter Menggunakan Algoritma Naïve Bayes. J-SAKTI (Jurnal Sains Komputer dan Informatika), 7(2), 802-814. doi:http://dx.doi.org/10.30645/j-sakti.v7i2.687
Rofiqi, M. A., Fauzan, Abd. C., Agustin, A. P., & Saputra, A. A. (2019). Implementasi term-frequency inverse document frequency (TF-IDF) untuk mencari relevansi dokumen berdasarkan query. ILKOMNIKA: Journal of Computer Science and Applied Informatics, 1(2), 58–64. https://doi.org/10.28926/ilkomnika.v1i2.18
Ruslim, K. I., Adikara, P. P., & Indriati, I. (2019). Analisis Sentimen Pada Ulasan Aplikasi Mobile Banking Menggunakan Metode Support Vector Machine dan Lexicon Based Features. Jurnal Pengembangan Teknologi Informasi Dan Ilmu Komputer, 3(7), 6694–6702. https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/5792
Sepri, D. (2020). Penerapan Algoritma Naïve Bayes Untuk Analisis Kepuasan Penggunaan Aplikasi Bank. Journal of Computer System and Informatics (JoSYC), 2(1), 135-139.
Shen, X., Chen, Z., Backes, M., & Zhang, Y. (2023). In chatgpt we trust? Measuring and characterizing the reliability of chatgpt. arXiv.Org. https://arxiv.org/abs/2304.08979
Santoso, V. I., Virginia, G., & Lukito, Y. (2017). Penerapan Sentiment Analysis Pada Hasil Evaluasi Dosen Dengan Metode Support Vector Machine. Jurnal Transformatika, 14(2), 72–76. https://doi.org/10.26623/transformatika.v14i2.439
Yusuf, L., & Masripah, S. (2023). Sentimen Analisis ChatGPT Dengan Algoritma Naive Bayes Dan Optimasi PSO. INTI Nusa Mandiri, 18(1), 59–64. https://doi.org/10.33480/inti.v18i1.4230
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Adji Surya Pamungkas, Nuri Cahyono
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Semua tulisan pada jurnal ini adalah tanggung jawab penuh penulis. Edumatic: Jurnal Pendidikan Informatika bisa diakses secara free (gratis) tanpa ada pungutan biaya, sesuai dengan lisensi creative commons yang digunakan.
This work is licensed under a Lisensi a Creative Commons Attribution-ShareAlike 4.0 International License.
