Implementation of Thermal Camera for Human Stress Detection: A Review

Stress has become a major problem that people face today. The high level of competition and environmental demands make people more susceptible to stress. Stress can interfere with a person’s ability to work effectively. If left unchecked for a long time, stress can cause various dangerous diseases such as hypertension, heart problems, and others that can lead to death. Research has been conducted for a long time to detect stress. Various technologies have been used to detect and anticipate stress that occurs in humans. One promising technology for detecting stress is the use of thermal cameras. Thermal cameras have several advantages: being non-contact and non-invasive, quick, easy to use, and cost-effective. In general, the architecture of the stress detection system using a thermal camera consists of several stages, including image acquisition, pre-processing, ROI tracking and selection, feature extraction, and statistical analysis or classification. This paper aims to review the use of thermal cameras in detecting stress in humans. This paper also seeks to answer the research question of what analysis can be done to improve stress detection accuracy using thermal camera images. Research shows that ROI selection must be carefully considered to obtain good accuracy. Combining thermal images with other data can improve accuracy in stress detection. Machine learning in classification provides many benefits in recognizing patterns but is highly influenced by the number of datasets used.


Introduction
Stress is a common occurrence in human existence.This condition can harm a person's health if they experience excessive physical or emotional stress.It is vital to detect stress quickly and effectively to prevent potential harm.Early stress detection can prevent negative consequences such as mental health issues, decreased performance, and increased injury risk.
The field of stress research continues to advance, including the application of thermal imaging technology to detect stress.Thermal imaging is a technology capable of recording and displaying changes in human body temperature.During times of stress, the body experiences changes in temperature in areas such as the face, neck, and back.This can be used as an indicator of stress.
Recent research indicates that thermal imaging technology can effectively detect stress, including in situations that trigger stress, such as the workplace, sports environments, and even daily life.Thermal cameras can be useful tools for recognizing human stress because they can detect changes in skin temperature and indicate stress levels.
Here are some advantages of using thermal cameras to recognize stress on humans: • Non-invasive: Thermal cameras do not require physical contact with the monitored person.This can be useful when the person may be uncomfortable with touching or where physical contact may interfere with the measurement.• Quick and Easy: Thermal cameras can provide a quick and easy way to measure stress levels.The camera can capture images in real-time, allowing for immediate analysis and feedback.• Objective Measurement: Thermal cameras objectively measure stress levels, which can be more reliable than subjective measures such as self-reported stress levels.• Cost-Effective: Thermal cameras can be cost-effective compared to other methods of measuring stress, such as blood tests or saliva samples.• Non-Verbal: Thermal cameras can measure stress levels without verbal communication.This can be useful in situations where the person may not be able to express their stress levels verbally Several studies have developed stress detection in humans using a thermal camera.Research [1,2] uses temperature from the human face to detect stress.A study by [3] can detect stress automatically in mobile conditions using thermal imaging.Research by [4,5] used machine learning to predict stress by analyzing images of the thermal human face.
This paper aims to review the use of thermal cameras in their application to stress detection.Compared to paper [6,7], we specifically review stress detection using a thermal camera [8,9,10,11,12,13].
Compared to previous review papers [14,15], we focused on comparing image acquisition and image processing on thermal images to detect stress in humans.This paper will also answer the research question: What types of analysis can be performed on stress detection data obtained using thermal cameras to identify human stress levels accurately?
We have divided this paper into several sections.In the I section, we provide an overview of the background of stress detection research using a thermal camera.Section II outlines a general stress detection system architecture that employs a thermal camera.Section III specifically discusses the use of thermal cameras for stress detection.In the IV section, we present several studies conducted to achieve our research objectives, providing an overview of the various types of thermal cameras, processing steps, and validation methods.Finally, we conclude with Section V.

System Architecture in General
Research to detect stress using a thermal camera is very diverse.Programming languages that are quite popular are Python [4] and Matlab [1,16].However, in general, the architecture of a stress detection system using a thermal camera can be summarized as shown in Figure 1.In contrast to research [14], we added image acquisition, as this step is crucial in the processing.Additionally, we included a statistical test alongside classification, as this section is also important.In addition to system architecture, other things that can be explained in stress detection using a thermal camera are the camera specifications, pre-processing techniques, detection and selection of Region of Interest (ROI), and feature extraction and validation of the methods used.We will explain further in the sub-section

Image Acquisition
Stress detection using a thermal camera requires camera sensitivity sensitive to temperature changes.Changes in heart activity when a person experiences stress will increase the temperature in parts of the human body, for example, the face.This temperature change can only be observed by a camera with high sensitivity.
Thermal camera specifications usually include high thermal resolution, the ability to record images at high speed, and the ability to measure temperature over long distances.Some brands of thermal cameras commonly used are FLIR [1,4,17,18,19] and Seek Thermal [20].
Thermal cameras must meet several essential requirements.In general, the cameras used in the included studies can record video at frame rates ranging from 8.7 FPS to 60 FPS.The relationship between FPS and resolution and the resulting signal quality [21,22] is close.FPS refers to the number of thermal image frames captured per second.The greater the number of image frames obtained, the greater the variation and variability of thermal information.In this instance, FPS can be interpreted as a system's sampling rate.Similarly, the image's dimensions indicate the number of measurements points the thermal camera makes.The larger the image dimensions, the more easily the system will detect the region of interest (ROI).In addition to thermal image specifications, temperature accuracy, and sensitivity are frequently used to describe the performance of thermal cameras.Temperature accuracy indicates how close a thermal imager measurement is to the actual absolute temperature.In contrast, thermal sensitivity is the noise equivalent temperature difference (NETD).This value indicates the smallest detectable temperature difference by the camera.Several studies [23] suggest that NETD is a crucial indicator of the performance of a thermal camera.The value of NETD is also an essential variable when using inexpensive thermal cameras for medical applications.For instance, a thermal camera with a NETD of less than 50 mK is ideal for medical applications [24].In studies involving the measurement of respiratory rate and heart rate, the value of the NETD becomes an important factor because the size of the respiratory rate takes temperature changes into account rather than the temperature's actual value.

Pre-Processing technique
A pre-processing technique commonly used in facial thermal image processing to prepare images for further analysis.This technique can be applied by resizing image dimensions, converting the number of FPS, converting thermal images to grayscale images, quantization, or histogram equalization.
Thermal image quality can be affected by thermal dynamics (i.e., different ambient temperatures and their dynamic variations).This problem causes tracking imperfections, which can change morphological properties.Quantization applied to thermal imaging sequence to negotiate thermal image quality problems caused by thermal dynamics.Quantization will assign continuous temperature range value to digital color-mapped equivalent.
Another image quality enhancement technique in thermal image preprocessing is using CLAHE (Contrast Limited Adaptive Histogram Equalization) to enhance an image.CLAHE is developed from histogram equalization.In CLAHE, the stretching histogram has a maximum limit value to obtain a better-intensity image because there is an uneven distribution of the pixels' intensity.

ROI tracking and selection
The selection of the region of interest (ROI) is crucial in detecting stress through thermal imaging of the face, as the ROI helps us focus the analysis on the areas of the face most responsive to temperature changes that occur due to stress.The human face has been widely selected as a local area for emotion recognition because it is extraordinarily sensitive to emotions as a part of the body [25].
Thermal imaging cameras can measure temperature differences on the skin's surface, indicating signs of stress on the face.When someone experiences stress, there is an increase in sympathetic nerve activity that causes vasoconstriction, or the narrowing of blood vessels in the front, which reduces blood and oxygen flow to specific face areas.This can cause temperature changes on the skin, which thermal imaging cameras can detect.
In detecting stress through facial thermal imaging, the appropriate ROI can help focus the analysis on the areas of the face most sensitive to temperature changes, such as the areas around the eyes, nose, and forehead.Selecting the appropriate ROI can also help reduce errors in stress detection, such as if the selected ROI is too large or too small.Furthermore, selecting the appropriate ROI can help increase data collection and analysis efficiency.By focusing the research on the areas most responsive to temperature changes, we can reduce the amount of data that needs to be analyzed and improve stress detection accuracy.
One of the methods used in ROI tracking and selection includes the YOLO Algorithm.Images or video frames are classified with a certain probability of whether an object is in that grid.Then, this algorithm predicts bounding boxes that indicate the position and size of the object within that grid [26].
Some studies presented Thermal Gradient Flow to enhance tracking.This algorithm computes the thermal-gradient magnitude map and obtains morphological shape distinction caused by motion artifacts or respiration dynamics.Thermal gradient flow can help accurately localize and extract facial features from thermal images.
The Viola-Jones algorithm is a Haar-like feature-based algorithm used to detect facial and non-face objects in thermal image processing [27].It consists of the following steps: Haar-like Feature Extraction, Integral Image, Adaboost Training, Cascade Classification, and Non-maximum Suppression (NMS).Adaboost Training selects a small subset of the most discriminative Haar-like features for detecting objects, while Cascade Classification uses a cascade of classifiers to reject non-object regions quickly.NMS compares the overlapping bounding boxes of detected areas and selects the one with the highest confidence score.
Faster R-CNN is a two-stage object detection algorithm developed by Shaoqing Ren et al. in 2015.It consists of a region proposal network (RPN) and a convolutional network for classifying and refining the proposals.The RPN generates bounding box proposals and predicts the likelihood of an object being present inside each anchor box.ROI pooling extracts features for each region proposal, fed into separate branches for classification and bounding box regression.Faster R-CNN is trained and supervised using labeled training data and optimized network parameters to minimize classification and regression losses [28].It has shown impressive performance in object detection tasks and has been widely adopted in various applications [29].

Feature Extraction, Statistical Analysis and Classification
Thermal images provide a unique view of the object's surface temperature or living being under consideration.This temperature distribution can be used to identify changes in the body that are indicative of stress.Feature extraction involves identifying the most informative aspects of the thermal image in detecting pressure.
One feature extraction method is to identify regions of interest (ROIs) in the thermal image.ROIs are specific areas of the idea that are more likely to show changes in temperature or blood flow indicative of stress.For example, the forehead or neck regions of the human body may show an increase in temperature due to stress-induced changes in blood flow.
Once the ROIs are identified, various image-processing techniques can be applied to extract features indicative of stress.These techniques include: • Texture analysis: This involves analyzing the temperature distribution patterns within the ROIs.The texture of the thermal image can be analyzed using techniques such as Gabor filters or local binary patterns to identify changes in temperature distribution indicative of stress.
• Statistical analysis: This involves analyzing the statistical properties of the temperature distribution within the ROIs.Statistical features such as mean, standard deviation, and skewness can be extracted from the thermal image to identify changes in temperature distribution indicative of stress.• Machine learning-based feature extraction involves using machine learning algorithms to identify features most informative in detecting stress.For example, deep learning-based feature extraction techniques such as convolutional neural networks (CNNs) can be trained to identify features indicative of stress.
Several features are commonly used to predict stress from thermal images.We will explain the list below: the extracted features as either indicative of stress or not.This step is critical to identify stress from thermal images accurately and can be done using various classification techniques such as logistic regression, decision trees, or support vector machines.

Thermal camera for stress detection on human
We compared several studies in the last five years based on the steps following the architecture, as shown in Figure 1.Summary of the thermal camera on detected stress on humans listed on the table 1.

Discussion
This paper provides an overview of studies on human stress detection using a thermal camera.When using thermal cameras to detect stress in humans, several types of analysis can be performed on the collected data to identify the stress level accurately.
From the results of the review that we conducted on several articles, we have made several important points in detecting stress in humans through thermal imaging, namely: • Temperature analysis: This type of analysis involves measuring the temperature of specific body regions, such as the forehead or cheeks, nasal, periorbital, or hands and fingers, which are known to show changes in temperature in response to stress.The temperature data can then be analyzed to determine the extent and pattern of temperature changes, which may correlate with the level of stress experienced by the individual.From the summary in table.We conclude that no single ROI (Region of Interest) on the human face is universally accepted as the most effective for predicting stress.• Pattern recognition analysis: This type of analysis involves using machine learning algorithms to recognize patterns in the temperature data that may indicate stress.These algorithms can be trained on large datasets of temperature data collected from individuals experiencing varying stress levels.Once introduced, the algorithms can be used to identify similar patterns of temperature changes in new data sets, thereby detecting the presence and level of stress in individuals.• Statistical analysis: This type of analysis involves using statistical methods to identify significant differences in temperature data between individuals experiencing different stress levels.For example, the temperature data of individuals experiencing high stress levels may show greater variability or a greater change rate than those experiencing low-stress levels.The statistical test used is a t-test, specifically the post hoc paired t-test.• Multimodal analysis: This type of analysis involves combining thermal imaging data with other types of data, such as heart rate or respiratory rate, to improve stress detection accuracy.By analyzing multiple data types simultaneously, it may be possible to identify more subtle changes in physiological responses to stress that thermal imaging may not detect

Conclusion
The analysis techniques used to identify the stress level accurately include temperature, pattern recognition, statistical, and multimodal analysis.There is no single ROI on the human face that is universally accepted as the most effective for predicting stress.Therefore, researchers should carefully consider any study's specific context and goals before selecting an ROI for stress detection.
Machine learning algorithms to recognize patterns have the advantage of being able to identify more subtle temperature changes that may not be detected by manual analysis.However, it requires large datasets for training the algorithms, and the accuracy of the technique depends on the quality of the data used Additionally, combining thermal imaging data with other data types may improve stress detection accuracy.Multimodal analysis combines thermal imaging data with different data types, such as heart or respiratory rates, to improve accuracy.However, it requires specialized equipment and may be more complicated to implement.

Figure 1 .
Figure 1.The architecture of a stress detection system using a thermal camera

•
Mean temperature: The mean temperature of a region of interest (ROI) in the thermal image can indicate stress.Stress-induced changes in blood flow can result in increased or decreased temperature, depending on the location and cause of stress.• Standard deviation: The standard deviation of the temperature distribution within an ROI can indicate stress.Increased variability in temperature distribution within an ROI can be indicative of stress.• Asymmetry: Asymmetry in the temperature distribution within an ROI can indicate stress.Stress-induced changes in blood flow can result in asymmetry in the temperature distribution within an ROI.• Texture: The texture of the temperature distribution within an ROI can indicate stress.Stress-induced changes in blood flow can result in changes in the surface of the temperature distribution within an ROI.• Blood flow: The blood flow pattern within an ROI can indicate stress.Stressinduced changes in blood flow can result in changes in the blood flow pattern within an ROI.• Pulse rate: The pulse rate of the object or living being under consideration can indicate stress.Increased pulse rate can show stress-induced changes in the autonomic nervous system.• Respiration rate: The respiration rate of the object or living being under consideration can indicate stress.Increased respiration rate can show stress-induced changes in the autonomic nervous system.• Classification: The final step involves using machine learning algorithms to classify

Table 1 .
Summary of Thermal Camera for Stress Detection on Human