Integrasi CNN VGG19 dalam Aplikasi Real-Time untuk Mendeteksi Ekspresi Konsumen sebagai Indikator Kepuasan

Arnold Robert; Ida Nurhaida

doi:10.29408/edumatic.v9i3.31280

Authors

Arnold Robert Program Studi Teknik Informatika, Universitas Pembangunan Jaya https://orcid.org/0009-0000-5798-0247
Ida Nurhaida Center of Urban Studies, Universitas Pembangunan Jaya https://orcid.org/0000-0002-8528-0632

DOI:

https://doi.org/10.29408/edumatic.v9i3.31280

Keywords:

cnn, facial expression, real-time detection, satisfaction level, vgg19

Abstract

Facial expressions are an important nonverbal indicator in measuring customer satisfaction levels. The objective of our research is to develop a real-time desktop application for detecting facial expressions as indicators of satisfaction using the Convolutional Neural Network (CNN) VGG19 architecture. The application development process employs the waterfall method, encompassing: needs analysis using the Japanese Female Facial Expression (JAFFE) dataset, comprising 213 grayscale images across seven emotional categories sourced from zenodo.org; followed by system design through format conversion, augmentation (rotation, flip), and mapping expressions into satisfaction categories, as well as Python-based application development with VGG19 model integration. The final stage involved black-box testing to verify functionality. Our findings resulted in a GUI-based desktop application using PySide capable of automatically detecting and displaying satisfaction levels in real-time without an internet connection. Validation results showed an accuracy of 89.39%, with the highest performance in the disgust and happiness classes and the lowest in the fear and sadness classes. The system successfully passed functional testing with expression detection functioning as designed, consistently displaying satisfaction mapping results. Our research demonstrates the potential of integrating CNN into efficient, objective, and flexible consumer satisfaction measurement across various operational conditions, making this application usable by businesses to monitor.

References

Aghababaeyan, Z., Abdellatif, M., Briand, L., & Bagherzadeh, M. (2023). Black-box testing of deep neural networks through test case diversity. IEEE Transactions on Software Engineering, 49(5), 3182-3204. https://doi.org/10.1109/TSE.2023.3243522

Akbar, A. T., Saifullah, S., & Prapcoyo, H. (2024). Klasifikasi ekspresi wajah menggunakan convolutional neural network. Jurnal Teknologi Informasi dan Ilmu Komputer, 11(6), 1399-1412. https://doi.org/10.25126/jtiik.2024118888

Aprelyani, S. (2023). Faktor-faktor yang mempengaruhi loyalitas pelanggan: Kualitas layanan dan kepuasan pelanggan (Tinjauan pustaka manajemen pemasaran). Journal of Health, Education and Social Media (JHESM), 3(1), 9-15. https://doi.org/10.38035/jhesm.v3i1

Bouzakraoui, M. S., Sadiq, A., & Alaoui, A. Y. (2020). Customer satisfaction recognition based on facial expression and machine learning techniques. Advances in Science, Technology and Engineering Systems Journal, 5(4), 594–599. https://doi.org/10.25046/aj050470

Cowen, A. S., & Keltner, D. (2021). Semantic space theory: A computational approach to emotion. Trends in Cognitive Sciences, 25(2), 124–136. https://doi.org/10.1016/j.tics.2020.11.004

Damanik, K., Sinaga, M., Sihombing, S., Hidajat, M., & Prakoso, O. S. (2024). Pengaruh kualitas layanan, kebijakan publik dan kepuasan pelanggan terhadap loyalitas pelanggan. Jurnal Manajemen Pendidikan Dan Ilmu Sosial (JMPIS), 5(2), 76–84. https://doi.org/10.38035/jmpis.v5i2

Farokhah, L. (2021). Perbandingan Metode Deteksi Wajah Menggunakan OpenCV Haar Cascade, OpenCV Single Shot Multibox Detector (SSD) dan DLib CNN. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 5(3), 609–614. https://doi.org/10.29207/resti.v5i3.3125

Handika, R., Pramuditha, A. Z., & Fadhli, M. (2023). Deteksi wajah dengan model pretrained VGG19 dan metode convolutional neural network. Sistemasi: Jurnal Sistem Informasi, 13(5), 1998–2007. https://doi.org/10.32520/stmsi.v13i5.4399

Ilmawati, F. I., Kusrini, K., & Hidayat, T. (2024). Optimizing facial expression recognition with image augmentation techniques: VGG19 approach on FERC dataset. Sinkron: Jurnal dan Penelitian Teknik Informatika, 8(2), 632–640. https://doi.org/10.33395/sinkron.v8i2.13507

Julianto, R., & Alamsyah, D. (2021). Pengenalan ekspresi wajah menggunakan metode SVM dengan transformasi Fourier dan PCA. Klik: Jurnal Ilmu Komputer, 2(1), 1–12. https://doi.org/10.56869/klik.v2i1.282

Khani, N. I., & Rakasiwi, S. (2025). Penerapan Convolutional Neural Network dengan ResNet-50 untuk klasifikasi penyakit kulit wajah efektif. Edumatic: Jurnal Pendidikan Informatika, 9(1), 217–225. https://doi.org/10.29408/edumatic.v9i1.29572

Melatisudra, R. J. L., Utomo, S., Sutjiningtyas, S., & Hernawati. (2024). Implementasi pengenalan ekspresi wajah dengan menggunakan metode Convolutional Neural Network dan OpenCV berbasis webcam. Journal of Computer System and Informatics (JoSYC), 6(1), 339–348. https://doi.org/10.47065/josyc.v6i1.6114

Muften, H. A., & Khayeat, A. R. H. (2025). A comparison of the VGG19, InceptionV3, NASNetMobile, and ResNet50 architectures for object classification in thermal images. Journal of Information Systems Engineering and Management, 10(17S), 2468-4376. https://doi.org/10.52783/jisem.v10i17s.2849

Nining, A. N., & Delfi, D. H. (2024). Pengaruh Customer Experience dan Kualitas Pelayanan Terhadap Minat Beli Ulang Memediasi Kepuasan Pelanggan. JEMSI (Jurnal Ekonomi, Manajemen, dan Akuntansi), 9(6), 2971–2979. https://doi.org/10.35870/jemsi.v9i6.2016

Prasetyawan, D., & Gatra, R. (2022). Model Convolutional Neural Network untuk Mengukur Kepuasan Pelanggan Berdasarkan Ekspresi Wajah. Jurnal Teknik Informatika dan Sistem Informasi, 8(3), 661–673. http://dx.doi.org/10.28932/jutisi.v8i3.5493

Prayoga, R. H. (2020). Analysis of unsteady-state temperature distribution in Python based Joglo build. International Journal of Emerging Trends in Engineering Research, 8(9), 5788–5793. https://doi.org/10.30534/ijeter/2020/142892

Rachmawati, N. L., & Fitriani, M. (2023). Pengukuran kepuasan pelanggan menggunakan metode Service Quality (SERVQUAL): Studi kasus PT Pos Indonesia Kota Metro. Jurnal Penelitian dan Aplikasi Sistem dan Teknik Industri (PASTI), 17(1), 79–89. https://doi.org/10.22441/pasti.2023.v17i1.008

Rachmawati, O., Barakbah, A., & Karlita, T. (2024). Programming language selection for the development of deep learning library. JOIV: International Journal on Informatics Visualization, 8(1), 434–441. https://doi.org/10.62527/joiv.8.1.2437

Saputra, D., & Nugroho, R. A. (2022). Pengenalan ekspresi emosi pada citra wajah menggunakan extreme machine learning studi kasus dataset publik JAFFE. MALCOM: Indonesian Journal of Machine Learning and Computer Science, 2(2), 19–27. https://doi.org/10.57152/malcom.v2i2.363

Saadon, J. R., Yang, F., Burgert, R., Mohammad, S., Gammel, T., Sepe, M., Rafailovich, M., Mikell, C. B., Polak, P., & Mofakham, S. (2023). Real‑time emotion detection by quantitative facial motion analysis. PLOS ONE, 18(3), e0282730. https://doi.org/10.1371/journal.pone.0282730