Hey guys! I've been busy lately interested in diagnosing medical images in deep learning, so I spent a few hours building this: A deep learning model that diagnoses Melanoma. This model performs quite good: around 94% accuracy.I've used pytorch as the main library. The code for this project is on a Jovian notebook, which you can visit here.

Downloading the Dataset

Using the opendatasets library, we download the dataset from kaggle. The opendatasets library is very useful as it downloads the data and decompresses the file.

Importing the necessary libraries

Pytorch is the main library for the project, though there are other helpful libraries that we used like matplotlib for plotting graphs.

Applying Transformations To The Dataset

Applying transformations is an absolute must! Pytorch can't just use normal image files, it uses tensors which are like matrices that it uses to make its operations. We also need to set the images to the same size and normalize the data to improve the efficiency, as normalization puts the numbers into a small range.

We also define a batch size, we divide the data into batches so that it can fit on the GPU's memory as we can't fit all the data on the GPU at once! Even if we could, the training process will be very slow.

Loading The Data

Here we load the data from our computer. train_dir, val_dir, test_dir is where our data is located. Our data is divided already for us, so we don't need to do it ourselves.

Viewing The Images

Here we define some useful functions that will help us view the images. denormalize() takes the image and converts it back to its normal RGB state, as normalizing the images has changed the values. show_batch() however, takes a batch of images and shows them as a grid.

Here we can see a batch of images from the training dataset. As we can see, there are 15 images, the batch size we defined.

Using A GPU

If we have a GPU (a graphics card) that is cuda-enabled, we can use it as its computation is quicker than CPUs. That's why people generally use GPUs in deep learning.

To use the GPU (if one exists), we need to define functions that will transfer all the data and our model (which we will soon define) to it.

get_default_device() returns the object for the device that is available. If a GPU is available, it'll return an object for it, if not, then it will return an object for the cpu. We will use that object to transfer the data to the device it represents and do the computation on it.

to_device() is the function that moves the tensors to the device we give it.

DeviceDataLoader class wraps a dataloader to move the data to the device.

Setting the Device/Hardware For The Computation

Here we see that we have a GPU. We store this object in a variable so that we can transfer our data and model to it.

Defining Functions For Accuracy/Statistics

These functions help use during the training step/validation step. These functions return the statistics during the development of the model.

The Fun Part: Defining Our Model

Here we define our model. We decided to use a ResNet50 but any model should do as long as it suits the data we have. You can try different model architectures and see how that impacts the performance. We use the to_device() function to put the model on the GPU. You should also note that we can't put the data on the CPU and the model on the GPU, it should be on either one of them.

Model Architecture

Here we see how the model is structured.

Transferring the Data to the GPU

Let's now move the data to the GPU using the DeviceDataLoader class.

Defining the fit function

The fit function is what fits our model to the data we have. Rather than using a regular fit function, here we add some improvments, like Learning Rate Scheduling, weight decay and gradient clipping. You will find blog posts on these soon!

Also we add the @torch.no_grad() decorator over the evaluate function, as we don't need to compute gradients during evaluation.

Testing The Model Before Fitting

Here we test the model on the training set before fitting. We can see that the model performs badly which is obvious as it hasn't been trained on the data. We get around 50% accuracy, which is what we should expect as all guesses are random and this is a binary classification problem.

Setting Parameters For The fit_one_cycle Function

These are the parameters that we will use to fit the model to the data.

Fitting the Model

At last! The model took around 35 minutes to train, though it looks like we got good results! Around 94% Accuracy in 15 epochs!

Evaluating the model on the test dataset

Testing on the test dataset got us similar results, over 93% in accuracy.

Plotting the loss vs epoch

We defined a function called plot_losses() that takes the history variable and plots the loss vs the number of epochs and as we can see, it is evident that the loss decrease by time in both the training set and the validation set.

Testing on single images

This function allows us to predict using the model if skin has melanoma or not.

As we can see, the label of this image is "Melanoma", and the predicted label is "Melanoma" too. Pretty Cool!

Let's try another one for "NotMelanoma".

Saving Our Model

After all this work, we should save our model so that we can use it or improve it later.

Conclusion

So, that's it! We were able to make a model that can diagnose melanoma which gained an accuracy of 94%! While doctors need to make tests on the tissue sample, all we do is take a picture and give it to the model. I hope you enjoyed this walk-through of my project. Don't forget to follow me on Linkedin and github.