Implementation of Thermal Camera for Human Stress Detection: A Review

Atika Hendryani; Vita Nurdinawati; Andy Sambiono

doi:10.62146/ijecbe.v1i2.28

Authors

Atika Hendryani Universitas Indonesia
Vita Nurdinawati Poltekkes Kemenkes Jakarta II
Andy Sambiono Poltekkes Kemenkes Jakarta II

DOI:

https://doi.org/10.62146/ijecbe.v1i2.28

Keywords:

stress detection, thermal image, image processing, affective computing

Abstract

Stress has become a major problem that people face today. The high level of competition and environmental demands make people more susceptible to stress. Stress can interfere with a person's ability to work effectively. If left unchecked for a long time, stress can cause various dangerous diseases such as hypertension, heart problems, and others that can lead to death. Research has been conducted for a long time to detect stress. Various technologies have been used to detect and anticipate stress that occurs in humans. One promising technology for detecting stress is the use of thermal cameras. Thermal cameras have several advantages: being non-contact and non-invasive, quick, easy to use, and cost-effective. In general, the architecture of the stress detection system using a thermal camera consists of several stages, including image acquisition, pre-processing, ROI tracking and selection, feature extraction, and statistical analysis or classification. This paper aims to review the use of thermal cameras in detecting stress in humans. This paper also seeks to answer the research question of what analysis can be done to improve stress detection accuracy using thermal camera images. Research shows that ROI selection must be carefully considered to obtain good accuracy. Combining thermal images with other data can improve accuracy in stress detection. Machine learning in classification provides many benefits in recognizing patterns but is highly influenced by the number of datasets used.

References

C. He, M. Mahfouf, and L. A. Torres-Salomao, “Facial temperature markers for mental stress assessment in human-machine interface (HMI) control system,” in ICINCO 2018 - Proceedings of the 15th International Conference on Informatics in Control, Automation and Robotics, 2018, vol. 2, pp. 21–28, doi: 10.5220/0006820700210028.

Y. Cho, N. Bianchi-Berthouze, M. Oliveira, C. Holloway, and S. Julier, “Nose Heat: Exploring Stress-induced Nasal Thermal Variability through Mobile Thermal Imaging,” in 2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019, 2019, pp. 566–572, doi: 10.1109/ACII.2019.8925453.

Y. Cho, “Automated mental stress recognition through mobile thermal imaging,” in 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, 2018, vol. 2018-Janua, pp. 596–600, doi: 10.1109/ACII.2017.8273662.

S. Vasavi, P. Neeharica, and B. Wadhwa, “Regression Modelling for Stress Detection in Humans by assessing most prominent Thermal Signature,” in 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference, IEMCON 2018, 2019, pp. 755–762, doi: 10.1109/IEMCON.2018.8614747.

F. Gioia, A. Greco, A. L. Callara, and E. P. Scilingo, “Towards a Contactless Stress Classification Using Thermal Imaging,” Sensors, vol. 22, no. 3, pp. 1–16, 2022, doi: 10.3390/s22030976.

M. Clinton et al., “Implementation of Thermal Camera for Non-Contact Physiological Measurement: A Systematic Review,” 2021, doi: 10.3390/s21237777.

F. Fardian, M. Mawarpury, K. Munadi, and F. Arnia, “Thermography for Emotion Recognition Using Deep Learning in Academic Settings: A Review,” IEEE Access, vol. 10, no. August, pp. 96476–96491, 2022, doi: 10.1109/ACCESS.2022.3199736.

S. Pourmohammadi and A. Maleki, “Stress detection using ECG and EMG signals: A comprehensive study,” Comput. Methods Programs Biomed., vol. 193, 2020, doi: 10.1016/j.cmpb.2020.105482.

I. Ladakis and I. Chouvarda, “Overview of biosignal analysis methods for the assessment of stress,” Emerg. Sci. J., vol. 5, no. 2, pp. 233–244, 2021, doi: 10.28991/esj-2021-01267.

R. K. Nath, H. Thapliyal, A. Caban-Holt, and S. P. Mohanty, “Machine Learning Based Solutions for Real-Time Stress Monitoring,” IEEE Consum. Electron. Mag., vol. 9, no. 5, pp. 34–41, 2020, doi: 10.1109/MCE.2020.2993427.

“Human Emotion Recognition- Review of Sensorsand MethodsAndrius Dzedzickis1, Art ̄uras Kaklauskas2and Vytautas Bucinskas.pdf.” .

F. Akbar, G. Mark, I. Pavlidis, and R. Gutierrez-Osuna, “An empirical study comparing unobtrusive physiological sensors for stress detection in computer work,” Sensors (Switzerland), vol. 19, no. 17, 2019, doi: 10.3390/s19173766.

A. R. Subhani, W. Mumtaz, M. N. B. M. Saad, N. Kamel, and A. S. Malik, “Machine learning framework for the detection of mental stress at multiple levels,” IEEE Access, vol. 5, pp. 13545–13556, 2017, doi: 10.1109/ACCESS.2017.2723622.

M. C. T. Manullang, Y. H. Lin, S. J. Lai, and N. K. Chou, “Implementation of thermal camera for non-contact physiological measurement: A systematic review,” Sensors, vol. 21, no. 23, 2021, doi: 10.3390/s21237777.

D. B. L. Arasu, A. S. A. Mohamed, N. I. R. Ruhaiyem, N. Annamalai, S. L. Lutfi, and M. M. Al Qudah, “Human Stress Recognition from Facial Thermal-Based Signature: A Literature Survey,” C. - Comput. Model. Eng. Sci., vol. 129, no. 3, pp. 1–20, 2021, doi: 10.32604/CMES.2021.016985.

C. He, M. Mahfouf, and L. A. Torres-Salomao, “Thermal Imaging for Psychophysiological State Detection in the Human-Machine Interface (HMI) Control System,” in Lecture Notes in Electrical Engineering, vol. 613, Department of Automatic Control and Systems Engineering, The University of Sheffield, Mappin Street, Sheffield, S1 3JD, United Kingdom, 2020, pp. 214–229.

A. Ravindran, G. G. Krishna, Sagara, and S. S., “A Comparative Analysis of Machine Learning Algorithms in Detecting Deceptive Behaviour in Humans using Thermal Images,” in 2019 International Conference on Communication and Signal Processing (ICCSP), Apr. 2019, pp. 0310–0314, doi: 10.1109/ICCSP.2019.8697911.

S. Nayak, S. K. Panda, and S. Uttarkabat, “A Non-contact Framework based on Thermal and Visual Imaging for Classification of Affective States during HCI,” in 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184), Jun. 2020, pp. 653–660, doi: 10.1109/ICOEI48184.2020.9142883.

K. Baran, “Stress detection and monitoring based on low-cost mobile thermography,” Procedia Comput. Sci., vol. 192, pp. 1102–1110, 2021, doi: 10.1016/j.procs.2021.08.113.

M. Kowalski, “A study on presentation attack detection in thermal infrared,” Sensors (Switzerland), vol. 20, no. 14, pp. 1–19, 2020, doi: 10.3390/s20143988.

E. Nowara and D. McDuff, “Combating the impact of video compression on non-contact vital sign measurement using supervised learning,” in Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019, 2019, pp. 1706–1712, doi: 10.1109/ICCVW.2019.00211.

A. Kirimtat, O. Krejcar, A. Selamat, and E. Herrera-Viedma, “FLIR vs SEEK thermal cameras in biomedicine: Comparative diagnosis through infrared thermography,” BMC Bioinformatics, vol. 21, no. Suppl 2, pp. 1–10, 2020, doi: 10.1186/s12859-020-3355-7.

D.-I. Kim, G. Kim, G.-H. Kim, and K. S. Chang, “Responsivity and Noise Evaluation of Infrared Thermal Imaging Camera,” J. Korean Soc. Nondestruct. Test., vol. 33, no. 4, pp. 342–348, 2013, doi: 10.7779/jksnt.2013.33.4.342.

E. Villa, N. Arteaga-Marrero, and J. Ruiz-Alzola, “Performance Assessment of Low-Cost Thermal Cameras for Medical Applications,” Sensors, vol. 20, no. 5, p. 1321, Feb. 2020, doi: 10.3390/s20051321.

C. Goulart et al., “Visual and thermal image processing for facial specific landmark detection to infer emotions in a child-robot interaction,” Sensors (Switzerland), vol. 19, no. 13, 2019, doi: 10.3390/s19132844.

Chaitanya, S. Sarath, Malavika, Prasanna, and Karthik, “Human Emotions Recognition from Thermal Images using Yolo Algorithm,” in Proceedings of the 2020 IEEE International Conference on Communication and Signal Processing, ICCSP 2020, 2020, pp. 1139–1142, doi: 10.1109/ICCSP48568.2020.9182148.

M. Kowalski, A. Grudzień, and W. Ciurapiński, “Detection of human faces in thermal infrared images,” Metrol. Meas. Syst., vol. 28, no. 2, pp. 307–321, 2021, doi: 10.24425/mms.2021.136609.

R. Girshick, “Fast R-CNN,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2015 Inter, pp. 1440–1448, 2015, doi: 10.1109/ICCV.2015.169.

S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137–1149, 2017, doi: 10.1109/TPAMI.2016.2577031.

D. Cardone et al., “Driver Stress State Evaluation by Means of Thermal Imaging: A Supervised Machine Learning Approach Based on ECG Signal,” Appl. Sci., vol. 10, no. 16, p. 5673, Aug. 2020, doi: 10.3390/app10165673.

J. Panasiuk, P. Prusaczyk, A. Grudzień, and M. Kowalski, “Study on facial thermal reactions for psycho-physical stimuli,” Metrol. Meas. Syst., vol. 27, no. 3, pp. 399–415, 2020, doi: 10.24425/mms.2020.134591.

S. Nayak, B. Nagesh, A. Routray, M. Sarma, and S. Uttarkabat, “Estimation of Depression Anxieties and Stress through Clustering of Sequences of Visual and Thermal Face Images,” in 2021 IEEE 18th India Council International Conference (INDICON), Dec. 2021, pp. 1–6, doi: 10.1109/INDICON52576.2021.9691610.

K. Hong, “Stress Assessment Using Facial image Imprint,” in 2021 5th International Conference on Vision, Image and Signal Processing (ICVISP), Dec. 2021, pp. 324–329, doi: 10.1109/ICVISP54630.2021.00064.

F. Gioia, A. Greco, A. L. Callara, and E. P. Scilingo, “Towards a Contactless Stress Classification Using Thermal Imaging,” Sensors, vol. 22, no. 3, 2022, doi: 10.3390/s22030976.