Implementasi Metode Convolutional Neural Network (CNN) Pada Sistem Deteksi Emosi dari Ekspresi Wajah Manusia dengan Aplikasi Android sebagai Antarmuka Pengguna

Muhammad Andika  Fadilla

doi:10.47747/jpsii.v5i4.1754

Authors

Muhammad Andika Fadilla Universitas Indo Global Mandiri Palembang

DOI:

https://doi.org/10.47747/jpsii.v5i4.1754

Keywords:

Android application, Convolutional Neural Network, emotion detection, facial expressions, fer2013

Abstract

Emotion detection through human facial expressions plays an important role in various fields, such as human-computer interaction, psychology, and artificial intelligence. This thesis describes the implementation of Convolutional Neural Network (CNN) for emotion detection system based on human facial expressions, with an Android application as the user interface. The dataset used to train the CNN model consists of fer2013 and muxspace, which includes thousands of human facial images with various expressions. The system development includes data preprocessing, CNN model training, model evaluation and optimization, and integration with the Android application. The results show that the generated model is capable of accurately identifying emotions from human facial expressions in realtime and can be used in various practical applications.

References

Asvial, M., Pandoyo, M.F.G., Arifin, A.S., 2020. Entropy-Based k Shortest-Path Routing for Motorcycles: A Simulated Case Study in Jakarta. International Journal of Advanced Computer Science and Applications 11. https://doi.org/10.14569/IJACSA.2020.0110758

Bah, I., Xue, Y., 2022. Facial expression recognition using adapted residual based deep neural network. Intelligence & Robotics 2, 72–88.

Baltrušaitis, T., Ahuja, C., Morency, L.-P., 2019. Multimodal Machine Learning: A Survey and Taxonomy. IEEE Trans Pattern Anal Mach Intell 41, 423–443. https://doi.org/10.1109/TPAMI.2018.2798607

Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.-P., 2018. OpenFace 2.0: Facial

Behavior Analysis Toolkit, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). pp. 59–66. h Bieńkiewicz, M.M.N., Smykovskyi, A.P., Olugbade, T., Janaqi, S., Camurri, A., Bianchi-Berthouze, N.,

Björkman, M., Bardy, B.G., 2021. Bridging the gap between emotion and joint action.

Neurosci Biobehav Rev 131, 806–833.

https://doi.org/https://doi.org/10.1016/j.neubiorev.2021.08.014

Biørn-Hansen, A., Grønli, T.-M., Ghinea, G., Alouneh, S., 2019. An Empirical Study of

Cross-Platform Mobile Development in Industry. Wirel Commun Mob Comput 2019, 5743892. https://doi.org/10.1155/2019/5743892

Chollet, F., 2021. Deep learning with Python. Simon and Schuster.

Chollet, F., 2016. Building powerful image classification models using very little data. Keras Blog 5, 90–95.

Corrigendum: Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements, 2019. . Psychological Science in the Public Interest 20, 165–166. https://doi.org/10.1177/1529100619889954

Delia, L., Thomas, P., Corbalan, L., Sosa, J.F., Cuitiño, A., Cáseres, G., Pesado, P.,

Development Approaches for Mobile Applications: Comparative Analysis of Features, in: Arai, K., Kapoor, S., Bhatia, R. (Eds.), Intelligent Computing. Springer International Publishing, Cham, pp. 470–484.

Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y., Yang, M.-H., Torr, P., 2021.

Res2Net: A New Multi-Scale Backbone Architecture. IEEE Trans Pattern Anal Mach Intell 43, 652–662. https://doi.org/10.1109/TPAMI.2019.2938758

Géron, A., 2022. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow. “ O’Reilly Media, Inc.”

Girshick, R., Donahue, J., Darrell, T., Malik, J., 2016. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep learning. MIT press. Goodfellow,

I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B.,

Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., Zhou, Y., Ramaiah, C., Feng,

F., Li, R., Wang, X., Athanasakis, D., Shawe-Taylor, J., Milakov, M., Park, J.,

Ionescu, R., Popescu, M., Grozea, C., Bergstra, J., Xie, J., Romaszko, L., Xu, B.,

Chuang, Z., Bengio, Y., 2016. Challenges in Representation Learning: A Report on Three Machine Learning Contests, in: Lee, M., Hirose, A., Hou,

Z.-G., Kil, R.M. (Eds.), Neural Information Processing. Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 117–124.

Hassouneh, A., Mutawa, A.M., Murugappan, M., 2020. Development of a Real- Time Emotion Recognition System Using Facial Expressions and EEG based on machine learning and deep neural network methods. Inform Med Unlocked 20, 100372.

https://doi.org/https://doi.org/10.1016/j.imu.2020.100372

He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep Residual Learning for Image

Recognition.

He, K., Zhang, X., Ren, S., Sun, J., 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV).

Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R., 2015. Improving neural networks by preventing co-adaptation of feature detectors.

Hossain, M.S., Muhammad, G., 2019. Emotion recognition using deep learning approach from audio–visual emotional big data. Information Fusion 49, 69– 78.

https://doi.org/https://doi.org/10.1016/j.inffus.2018.09.008

Hu, D., 2020. An Introductory Survey on Attention Mechanisms in NLP Problems, in: Bi Yaxin and Bhatia, R. and K.S. (Ed.), Intelligent Systems and Applications. Springer International Publishing, Cham, pp. 432–448.

Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., 2017. Densely Connected Convolutional Networks.

Ioannidou, A., Chatzilari, E., Nikolopoulos, S., Kompatsiaris, I., 2017. Deep Learning Advances in Computer Vision with 3D Data: A Survey. ACM Comput. Surv. 50. https://doi.org/10.1145/3042064

Kanade, T., Cohn, J.F., Tian, Y., 2015. Comprehensive database for facial expression analysis, in: Proceedings Fourth IEEE International Conference on Automatic Face and

Gesture Recognition (Cat. No. PR00580). pp. 46–53. https://doi.org/10.1109/AFGR.2000.840611

Khaireddin, Y., Chen, Z., n.d. Facial Emotion Recognition: State of the Art Performance on FER2013.

Khalil, R.A., Jones, E., Babar, M.I., Jan, T., Zafar, M.H., Alhussain, T., 2019.

Speech Emotion Recognition Using Deep Learning Techniques: A Review. IEEE Access 7, 117327–117345.

https://doi.org/10.1109/ACCESS.2019.2936124

Ko, B.C., 2018. A brief review of facial emotion recognition based on visual information. sensors 18, 401.

Krizhevsky, A., Sutskever, I., Hinton, G.E., 2017. ImageNet Classification with Deep

Convolutional Neural Networks. Commun. ACM 60, 84–90. https://doi.org/10.1145/3065386

LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521, 436–444. https://doi.org/10.1038/nature14539

Lecun, Y., Bottou, L., Bengio, Y., Haffner, P., 2018. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324. https://doi.org/10.1109/5.726791

Lewis-Beck, M., Bryman, A.E., Liao, T.F., 2003. The Sage encyclopedia of social science research methods. Sage Publications.

Li, S., Deng, W., 2022. Deep Facial Expression Recognition: A Survey. IEEE Trans Affect Comput 13, 1195–1215. https://doi.org/10.1109/TAFFC.2020.2981446

Liu, J., Feng, Y., Wang, H., 2021. Facial Expression Recognition Using Pose- Guided Face Alignment and Discriminative Features Based on Deep Learning. IEEE Access 9, 69267–69277. https://doi.org/10.1109/ACCESS.2021.3078258

Lopes, A.T., de Aguiar, E., De Souza, A.F., Oliveira-Santos, T., 2017. Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order. Pattern Recognit 61, 610–628.

https://doi.org/https://doi.org/10.1016/j.patcog.2016.07.026

Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I., 2016. The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops. pp. 94–101. https://doi.org/10.1109/CVPRW.2010.5543262

Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J., 2017. Coding facial expressions with

Gabor wavelets, in: Proceedings Third IEEE International Conference on Automatic

Face and Gesture Recognition. pp. 200–205. https://doi.org/10.1109/AFGR.1998.670949

M. Saleem Abdullah, S., Abdulazeez, A.M., 2021. Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review. Journal of Soft Computing and Data Mining 2, 53–65.

MeriemSari AbdenourHadid, A., 2020. Automated Facial Expression Recognition Using Deep Learning Techniques: An Overview. Journal 3, 39–53.

Mollahosseini, A., Chan, D., Mahoor, M.H., 2016. Going deeper in facial expression recognition using deep neural networks, in: 2016 IEEE Winter Conference on

Applications of Computer Vision (WACV). pp. 1–10. https://doi.org/10.1109/WACV.2016.7477450

Picard, R.W., Vyzas, E., Healey, J., 2001. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23, 1175–1191. https://doi.org/10.1109/34.954607

Rouast, P. V, Adam, M.T.P., Chiong, R., 2021. Deep Learning for Human Affect Recognition: Insights and New Developments. IEEE Trans Affect Comput 12, 524–543. https://doi.org/10.1109/TAFFC.2018.2890471

Saurav, S., Gidde, P., Saini, R., Singh, S., 2022. Dual integrated convolutional neural network for real-time facial expression recognition in the wild. Vis Comput 38, 1083–1096. https://doi.org/10.1007/s00371-021-02069-7

Shorten, C., Khoshgoftaar, T.M., 2019. A survey on Image Data Augmentation for Deep Learning. J Big Data 6. https://doi.org/10.1186/s40537-019-0197-0

Sievi-Korte, O., Richardson, I., Beecham, S., 2019. Software architecture design in global software development: An empirical study. Journal of Systems and Software 158, 110400. https://doi.org/https://doi.org/10.1016/j.jss.2019.110400

Tang, H., Liu, W., Zheng, W.-L., Lu, B.-L., 2017. Multimodal Emotion Recognition Using Deep Neural Networks, in: Liu, D., Xie, S., Li, Y., Zhao,

D., El-Alfy, E.-S.M. (Eds.), Neural Information Processing. Springer International Publishing, Cham, pp. 811–819.

Tavakol, M., Dennick, R., 2011. Making sense of Cronbach’s alpha. Int J Med Educ 2, 53.

Zhang, J., Yin, Z., Chen, P., Nichele, S., 2020. Emotion recognition using multi- modal data and machine learning techniques: A tutorial and review.

Information Fusion 59, 103–126.

https://doi.org/https://doi.org/10.1016/j.inffus.2020.01.011

Zhang, Zhao, Tang, Z., Wang, Y., Zhang, Zheng, Zhan, C., Zha, Z., Wang, M., 2021. Dense Residual Network: Enhancing global dense feature flow for character recognition. Neural Networks 139, 77–85.

https://doi.org/https://doi.org/10.1016/j.neunet.2021.02.005