Related work

A comprehensive review of relevant literature was undertaken to learn about the current body of research. The following content summarizes the scholarly articles about the relevant research.

In one study, the authors conducted a widespread examination of the existing literature on the utilization of artificial intelligence (AI) in monitoring DFUs [15]. The study discussed the advantages of these techniques and highlighted the challenges associated with integrating them into a functional and dependable structure for effective management of remote patients. This research examines the imaging methods and optical sensors used to detect diabetic foot ulcers. The investigation takes into account the characteristics of the sensors as well as the physiological aspects of the patient. The data source supported various monitoring strategies, which imposed constraints on AI technologies.

In another study [13], a unique CNN model was created to classify healthy and abnormal skin automatically. This CNN was trained using a dataset of 754 colour photographs of DFUs and normal skin samples obtained from the diabetes centre at Nasiriyah Hospital. An expert in DFU manually annotated the database in order to create the ML model. The region of interest (ROI) dimensions were adjusted to 224 × 224 pixels. This modification resulted in a substantial area encompassing the ulcer, including crucial tissues from the normal and DFU categories. A trained expert then proceeded to annotate the clipped patches. The researchers used the DFU_QUTNet classifier to evaluate the performance of their model. The acquired F1-score was 93.24%. Additionally, they integrated network characteristics with an support vector machine (SVM) classifier, resulting in an improved performance of 94.5%.

Researchers [16] proposed the use of a non-invasive photonic-based apparatus as a potential treatment modality for addressing DFUs in patients diagnosed with problem of diabetes. The apparatus used the principles of hyperspectral and thermal imaging to evaluate the condition of a foot ulcer. The biomarkers of oxyhaemoglobin and deoxyhaemoglobin were estimated using photonic based imaging. By using super-resolution methods, the apparatus underwent enhancements via the incorporation of signal processing technologies that use deep learning algorithms to increase pixel accuracy and reduce noise.

In a study [17], the authors proposed using a specialized network called Dfu_SPNet, which is constructed using parallel stacked convolution layers, to classify the data of ulcer images. Dfu_SPNet used parallel convolution layers with three distinct kernel sizes in the context of feature abstractions. Dfu_SPNet achieved a superior performance compared to the current state-of-the-art results, as shown by its area under the curve (AUC) score of 97.4%. This notable improvement was seen after training the Dfu_SPNet model on the dataset, using the stochastic gradient descent (SGD) optimizer with a learning rate of 1e-2.

In a study [18], DFU classification using CNN was performed, and the researchers used RGB, i.e. 3 channel images, and corresponding information of texture as inputs for the proposed CNN model. The researchers found that the suggested framework exhibits superior classification performance compared to conventional RGB-based CNN models. The study’s first phase involved using the mapped binary pattern approach to gather the texture information of the RGB image. The mapped image generated from the ulcer texture information was used in the second step to aid in identifying DFUs. The CNN received a tensor input that is called a fused image, which is composed of a linear combination of the RGB and mapped binary pattern. The model achieved an F1 score of 95.2%.

In a further investigation [19], the authors used a dataset consisting of 50 thermograms captured at a distance of one foot. The classification task was carried out using a traditional machine-learning methodology. The intensity values of thermograms from both normal and diabetic feet were used to generate grey texture features. These features were then utilized as the input for the classifier model SVM. The accuracy of classification was determined to be 96.42%.

In a study [20], the authors used 33 thermographic images of the plantar foot region, including persons without any medical conditions and those diagnosed with type 2 diabetes. The decomposition of foot images was done using the discrete wavelet transform (DWT) with techniques of higher order spectra (HOS). The generated decomposed images were further used to find different texture and entropy attributes. The SVM classifier achieved a 81.81% sensitivity score, 89.39% accuracy, and a specificity of 96.97% based on a limited set of five features.

The authors of another study [21] devised an innovative skin tele-monitoring system that utilizes a smartphone to improve the medical diagnosis and decision-making process in assessing DFU tissue. In order to accurately identify tissues and label the ground facts, a dataset consisting of 219 photos was used. A graphical interface was also developed based on the super pixel segmentation technique. The system’s method included a thorough examination of DFU, including automated ulcer segmentation and categorization. The retrieved super pixels serve as the training input for the deep neural network, and classification at the patch level was performed. The model under consideration had an accuracy of 92.68% and a DICE score of 75.74%. An alternative methodology [22] extracted features from the CNN model known as Densenet201. These features were then inputted into a SVM classifier to facilitate the diagnosis of DFUs.

Several other studies have examined the location of infection in conjunction with its categorization. In a study [23], a CNN architecture consisting of 16 layers was presented for the purpose of classifying infection/ischemia models. The process included extracting deep features, which were then used as the input for a range of machine learning classifiers in order to assess the performance of the model evaluation metrics. In addition, the researchers used gradient-weighted class activation mapping (Grad-CAM) to visually represent the prominent characteristics of an infected zone, hence facilitating a more comprehensive interpretation after the classification process. The photos were inputted into a YOLOv2-DFU network to identify and localize affected regions. The suggested model demonstrated an accuracy of 99% for infection classification and 97% for ischemia classification. In order to identify and pinpoint areas of the foot that are affected by unhealthy conditions, the researchers achieved an intersection over union (IOU) score of 0.98 for ischemia and 0.97 for infection.

In other research [24], a thorough examination of model scaling was undertaken, revealing that the meticulous optimization of network breadth, depth, and resolution may achieve performance enhancements. Based on the result above, a unique scaling approach was proposed, which used a simple but very effective compound coefficient to uniformly scale all three parameters. A baseline network was developed using neural architecture and then expanded to generate the EfficientNets series of models. These models show superior performance compared to previous convolutional neural networks (ConvNets) in terms of effectiveness and accuracy. Additionally, they exhibited the advantages of smaller size and quicker inference speed. Notably, the EfficientNets achieved an accuracy rate of 84.3% on the ImageNet dataset.

Furthermore, there is a limited body of research that has used thermal imaging of the foot, since the majority of studies have relied on visual foot images that are vulnerable to fluctuations in the surrounding environment. In order to address these constraints, we have devised a dependable and entirely automated computer-aided design compatible model based on DL techniques for the purpose of diagnosing DFUs. Thermal imaging was selected as the preferred method for monitoring the distribution of temperature over the foot, with the aim of aiding in the timely detection of DFUs. In addition, the thermal images exhibit reduced susceptibility to the influence of the surrounding environment. In addition, before testing the model for better predictions and accuracy the pre-processing techniques canny edge detection and watershed segmentation were applied and a pre-processed dataset was created. Canny edge detection highlights the edges in an image, providing important features for subsequent analysis. These edges can capture important structures and boundaries in the image, which can be useful for classification tasks [8]. Edge detection can also help reduce noise in the image by emphasizing only the most significant edges. This can also improve the quality of the input data for classification algorithms, leading to more accurate results. Watershed segmentation can partition the image into regions based on gradients and intensities, effectively separating different objects or regions of interest [25]. This segmentation can help isolate individual objects or areas within the image, making it easier for the classifier to focus on relevant features.

Dataset

A thermogram image dataset was used for this study [14]. The dataset consists of two type of images – normal and ulcer. In total, 1055 images are present in the dataset; among these images, 543 are normal and the rest are of ulcers. A sample of normal feet images from dataset is shown in Figure 1, while Figure 2 shows sample images of abnormal feet from the dataset. Both types of files are organized in separate folders named normal and ulcer respectively and labelled with numbering.

Figure 1

Sample images of normal feet from the dataset

Figure 2

Sample images of abnormal feet from the dataset

Canny edge detection

The images in the dataset were transformed using the canny edge detection technique. Canny edge detection is a technique widely used in computer vision for detecting edges in images. Since its creation in 1986 by John F. Canny, it has evolved into one of the industry norms for edge detection. The basic steps involved in the process are as follows: The input image is smoothed, Eq. 1 represents the equation of the 2D Gaussian function, and edges of the image are detected using the Sobel filter. The Sobel operator calculates the gradient G_x in the x direction and G_y in the y direction using the kernel given in Eq. 2 and 3. After this, suppression of type non-max is applied and the local maximum pixels in the gradient direction are reserved; the rest of the pixels are blocked. Eq. 4 and Eq. 5 represent the gradient magnitude and gradient direction respectively. The thresholding is applied to remove the pixels which are less than a certain threshold value and pixels are considered if above a certain threshold value to remove the edges that could be formed due to noise. Finally, hysteresis tracking is applied with the two thresholds T_high and T_low to classify pixels as strong, weak and no edge [8, 26].

Where F(x, y) is a Gaussian kernel at position (x,y), standard deviation is represented as σ of the Gaussian distribution, e is the base for the logarithm, and the coordinates are x and y in the Gaussian kernel.

Where GM(x, y) represents gradient magnitude and θ(x, y) represents gradient direction.

Watershed segmentation

The watershed algorithm is a method of separating images that relies on watershed transformation. The segmentation method relies on the relative similarity of neighbouring pixels in the image as a crucial point of reference for linking pixels with similar spatial coordinates and grey values.

The watershed method uses topographic information to partition an image into distinct parts. The method considers the image as a topographic surface and uses pixel intensity to locate catchment basins. Starting locations are designated local minima, and catchment basins are filled with colours until the object’s limits are reached. The segmentation process gives distinct colours to certain areas, facilitating the identification of objects and the study of images [25, 27].

The whole procedure of the watershed algorithm may be succinctly outlined via the following steps:

Data augmentation

In order to operate efficiently, deep learning model requires a substantial amount of labelled training data. In addition, gathering a large amount of medical data is costly and difficult. In order to enhance the performance of the deep learning model and avoid the problem of overfitting, the data augmentation technique is used. Various image processing methods are used for data augmentation to achieve the intended outcomes, such as resizing, rotation, and translation.

Resizing – Through this dimensions of the image are changed. Let I₀ be the original image with dimensions (H₀, W_o, C₀) , where H₀ is height, W_o is width and C₀ is number of channels. Resizing is calculated by Eq. 6.

Where H0', W0' are the new height and width respectively.

Random Rotation – Let I₀ be the original image. Random rotation is calculated by Eq. 7.

Where θ is a random rotation angle.

Random Translation – Let I₀ be the original image. Random rotation is calculated by Eq. 8.

Where t_x, t_y are random translation amounts in the horizontal and vertical directions respectively.

Random Flip – Let I₀ be the original image. Random flipping is calculated by Eq. 9. The flip operation can be horizontal or vertical, which will be chosen randomly.

Gaussian Noise – Let I₀ be the original image. Gaussian noise is calculated by Eq. 10.

Where μ is the mean and σ is standard deviation of the Gaussian distribution.

Random Contrast – Let I₀ be the original image. Random contrast is calculated by Eq. 11.

Where C_f represents the random contrast factor.

Proposed model

This section provides an overview of the deep learning models used for the prediction of foot ulcer instances. The model is trained using a regression classifier utilising a scaled and filtered image dataset, after feature extraction using pre-trained models and filters. Figure 3 displays the block diagram of the suggested model, whereas Algorithm 1 illustrates the significant processes involved in the proposed model. The approach under consideration was assessed using the whole dataset, including both normal and diseased images. In the proposed method a new dataset is created by applying the canny edge detection and watershed segmentation on the images present in the dataset. In the first step canny edge detection is applied to all the images of the normal category and the abnormal category. After edge detection the watershed marker segmentation is applied on the images obtained after edge detection. All the converted images are subsequently stored in the two folders converted_normal and converted_abnormal. Figure 4 shows an example image obtained after each step of the edge detection and segmentation. The output image obtained is the image of the new dataset. After creation of the new dataset, the data augmentation with six methods is applied to accomplish two goals – one to increase the size of the dataset for training of the models and another to balance the dataset as 543 images are normal and 512 are abnormal.

Figure 3

Block diagram of proposed classification model

Figure 4

Sample of image obtained after edge detection and segmentation

The following part provides a comprehensive analysis of the training, testing, and other factors. The classification model utilises the Adam optimizer, which is based on SGD. The primary differentiating factor between Adam optimizers and SGD is in their updating processes. SGD calculates gradients as per a single training instance randomly to update the model’s parameters. In contrast, Adam combines the advantages of adaptive learning rates and estimates of first- and second-order moments of the gradients. This leads to more effective and efficient parameter updates. The learning rate in the Adam optimizer is adjusted dynamically during training by using averages of the parameters. Eq. 12 illustrates the loss function. Table 1 illustrates the detailed parameters of the model.

Where, at time t , G_t represents the gradient of the loss function, the gradient operator is represented by ∇ , θ_t – model parameters at time t , F_L – loss function and D_train – training data. The first and second moments of the gradients are initialized by 0 and represented as M₀ , V₀ = 0. Eq. 13 is used to update the first moment.

Where γ₁ represents the exponential decay rate. The second moment estimate is updated using Eq. 14.

Where γ₂ represents the exponential decay rate. Eq. 15 and 16 are used for calculation of bias.

Model parameters are updated using Eq. 17

Where learning rate is represented by α and τ is a small constant.

In the proposed models sparse categorical cross entropy is used to measure the divergence between the predicted probabilities and actual probabilities of the dataset. Eq. 18 shows the cross entropy for single training. Eq. 19 shows the average cross-entropy loss over the entire dataset.

Where L_i is the loss for the i^th training sample, p_i,yi is the logit corresponding to class j for the i^th training sample. y_i is the true label for the i^th training sample and C is the total number of classes.

Where N is the total number of samples.

Algorithm 1: