An Exploratory Data Analysis for Synchronous Online Learning Based on AFEA Digital Images
Abstract
The spread of COVID-19 throughout the world has affected the education sector. In some higher education institution, such as Polytechnic Caltex Riau (PCR), it is mandatory for students to participate in synchronous or asynchronous learning activities via virtual classroom. Synchronous online learning is usually supported by video conferencing media such as Google Meeting or Zoom Meeting. The communication between lecturers and students is captured as an image as evidence of students’ interaction and participation in certain learning subjects. These images can provide information for lecturers in determining students’ internal feelings and measuring students’ interest through facial emotions. Taking this reason into account, the current research aims to analyze the emotions detected in facial expression through images using automatic facial expression analysis (AFEA) and exploratory data analysis (EDA), then visualize the data to determine the possible solution to improve the educational process’ sustainability. The AFEA steps applied were face acquisition to detect facial parts in an image, facial data extraction and representation to process feature extraction on the face, and facial expression recognition to classify faces into emotional expressions. Thus, this paper presents the results obtained from applying machine learning algorithms to classify facial expressions into happy and unhappy emotions with mean values of 5.58 and 2.70, respectively. The data were taken from the second semester of 2020/2021 academic year with 1,206 images. The result highlighted the fact that students showed the facial emotion based on the lecture types, hours, departments, and classes. It indicates that there are, in fact, several factors contributing to the variances of students’ facial emotions classified in synchronous online learning.
References
S. Hrastinki, “Asynchronous and Synchronous Cooperation,” Educause Quart., Vol. 4, pp. 51-55, 2008.
Z. Zayapragassarazan, “COVID-19: Strategies for Online Engagement of Remote Learners,” 2020.
V. Gherheș, S. Șimon, and I. Para, “Analysing Students’ Reasons for Keeping Their Webcams On or Off during Online Classes,” Sustainability, Vol. 13, No. 6, pp. 1-13, Mar. 2021.
M. Sathik and S.G. Jonathan, “Effect of Facial Expressions on Student’s Comprehension Recognition in Virtual Educational Environments,” Springerplus, Vol. 2, pp. 1–9, 2013.
B. Giesbers, B. Rienties, D. Tempelaar, and W. Gijselaers, “Investigating the Relations Between Motivation, Tool Use, Participation, and Performance in an E-Learning Course Using Web-Videoconferencing,” Comput. Human Behav., Vol. 29, No. 1, pp. 285–292, 2013.
V. Gherheș, C.E. Stoian, M.A. Fărcașiu, and M. Stanici, “E-Learning vs. Face-to-Face Learning: Analyzing Students’ Preferences and Behaviors,” Sustainability, Vol. 13, No. 8, pp. 1-15, Apr. 2021.
W. Wang, K. Xu, H. Niu, and X. Miao, “Emotion Recognition of Students Based on Facial Expressions in Online Education Based on the Perspective of Computer Simulation,” Complexity, Vol. 2020, pp. 1–9, Sep. 2020.
J. Shen, H. Yang, J. Li, and Z. Cheng, “Assessing Learning Engagement Based on Facial Expression Recognition in MOOC’s Scenario,” Multimed. Syst., Vol. 28, pp. 469-478, Oct. 2021.
D. Canedo and A.J.R. Neves, “Facial Expression Recognition Using Computer Vision: A Systematic Review,” Appl. Sci., Vol. 9, No. 21, pp. 1-31, Nov. 2019.
G. Tonguç and B.O. Ozkara, “Automatic Recognition of Student Emotions from Facial Expressions During a Lecture,” Comput. Educ., Vol. 148, pp. 1-12, Apr. 2020.
C. Pabba and P. Kumar, “An Intelligent System for Monitoring Students’ Engagement in Large Classroom Teaching Through Facial Expression Recognition,” Expert Syst., Vol. 39, No. 1, pp. 1-28, Jan. 2022.
“Kewaspadaan and Pencegahan Penyebaran Covid-19 di Lingkungan Politeknik Caltex Riau,” Surat Edaran Direktur No. 003/Dir/PCR/2020, 2020.
C. Chotimah, “Hubungan Kestabilan Emosi dengan Prestasi Belajar Pada Siswa Kelas X di SMA Negeri 1 Karanganom Klaten,” Undergraduate thesis, Universitas Sebelas Maret, Surakarta, Indonesia, 2010.
N. Fitriansyah, “Deskripsi Kemampuan Mengenal Emosi dan Mengelola Emosi Pada Siswa Kelas X SMA Pangudi Luhur Sedayu Yogyakarta Tahun Ajaran 2007/2008 dan Implikasinya Terhadap Usulan Topik Bimbingan Klasikal,” Undergraduate thesis, Universitas Sanata Dharma, Yogyakarta, Indonesia, 2008.
M.A.A. Dewan, M. Murshed, and F. Lin, “Engagement Detection in Online Learning: A Review,” Smart Learn. Environ., Vol. 6, No. 1, pp. 1-20, Dec. 2019.
Z. Song, “Facial Expression Emotion Recognition Model Integrating Philosophy and Machine Learning Theory,” Front. Psychol., Vol. 12, pp. 1-9, Sep. 2021.
P. Tarnowski, M. Kołodziej, A. Majkowski, and R.J. Rak, “Emotion Recognition Using Facial Expressions,” Procedia Comput. Sci., Vol. 108, pp. 1175–1184, 2017.
H. Husdi, “Pengenalan Ekspresi Wajah Pengguna Elearning Menggunakan Artificial Neural Network dengan Fitur Ekstraksi Local Binary Pattern dan Gray Level Co-Occurrence Matrix,” ILKOM. J. Ilm., Vol. 8, No. 3, pp. 212–219, 2016.
K. Wolf, “Measuring Facial Expression of Emotion,” Dialogues Clin. Neurosci., Vol. 17, No. 4, pp. 457–462, 2015.
Y.-L. Tian, T. Kanade, and J.F. Cohn, “Facial Expression Analysis,” dalam Handbook of Face Recognition, S.Z. Li and A.K. Jain, Eds., New York, USA: Springer-Verlag, 2005, pp. 247–275.
B. Heiselet, T. Serre, M. Pontil, and T. Poggio, “Component-based Face Detection,” Proc. 2001 IEEE Comput. Soc. Conf. Comput. Vis., Pattern Recognit. (CVPR 2001), 2001, Vol. 1, pp. I-657-I-662.
S.Z. Li, et al., “Real-Time Multi-View Face Detection, Tracking, Pose Estimation, Alignment, and Recognition,” IEEE Conf. Comput. Vision, Pattern Recognit. Demo Summary, 2001, pp. 1-2.
Y.B. Kim, et al., “Efficiently Detecting Outlying Behavior in Video-Game Players,” PeerJ, Vol. 3, pp. 1-19, Dec. 2015.
S. Bayrakdar, D. Akgün, and İ. Yücedağ, “A Survey on Automatic Analysis of Facial Expressions,” Res. Artic., Vol. 20, No. 2, pp. 383-398, Dec. 2016.
R.E. Kleck and M. Mendolia, “Decoding of Profile Versus Full-Face Expressions of Affect,” J. Nonverbal Behav., Vol. 14, No. 1, pp. 35–49, 1990.
M. Mukhopadhyay, et al., “Facial Emotion Detection to Assess Learner’s State of Mind in an Online Learning System,” 5th Int. Conf. Intell. Inf. Technol. (ICIIT 2020), 2020, pp. 107–115.
U. Ayvaz, H. Gürüler, and M.O. Devrim, “Use of Facial Emotion Recognition in E-Learning Systems,” Inf. Technol., Learn. Tools, Vol. 60, No. 4, pp. 95-104, 2017.
P. Utami, R. Hartanto, and I. Soesanti, “A Study on Facial Expression Recognition in Assessing Teaching Skills: Datasets and Methods,” Procedia Comput. Sci., Vol. 161, pp. 544–552, 2019.
S. Yulina, “Penerapan Haar Cascade Classifier dalam Mendeteksi Wajah dan Transformasi Citra Grayscale Menggunakan OpenCV,” J. Politek. Caltex Riau, Vol. 7, No. 1, pp. 100–109, 2021.
© Jurnal Nasional Teknik Elektro dan Teknologi Informasi, under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
1.png)
