Asia Pacific Tele-Ophthalmology Society (APTOS) aims to bring together clinicians, researchers, technicians, institutes and organisations to form an alliance that promotes communication, exchange and collaboration in Tele-ophthalmology. It provides a platform on which eye care or Tele-medical professionals can share knowledge and collaborate to deliver efficient, accessible and quality universal eye care throughout the region.
Diabetic Retinopathy(DR) is an eye disease associated with long-standing diabetes. Millions of people suffer from diabetic retinopathy, the leading cause of blindness among working-aged adults. Detecting DR is a time-consuming and manual process that requires a trained clinician to examine and evaluate digital colour fundus photographs of the retina. Aravind Eye Hospital in India hopes to detect and prevent this disease among people living in rural areas where medical screening is difficult to conduct. Aravind technicians travel to these rural areas to capture images and then rely on highly trained doctors to review the images and provide a diagnosis. Their goal is to scale their efforts through technology to gain the ability to automatically screen images for disease and provide information on how severe the condition may be. We need to work with thousands of images collected in rural areas to help identify diabetic retinopathy automatically
Problem Statement:The problem statement is inspired from Kaggle problem APTOS 2019 Blindness Detection. To design a Neural Network model which gives higher precision in detecting the level of blindness the patient is encountering. A clinician has rated each image for the severity of diabetic retinopathy on a scale of 0 to 4.
0 – No DR
1 – Mild
2 – Moderate
3 – Severe
4 – Proliferative DR
Data Sources:2 Kaggle competitions held to detect Diabetic Retinopathy as below.
Diabetic Retinopathy Detection which held in 2015 provided with 35K Images .
APTOS 2019 Blindness Detection which held in 2019 provided with 3K Images.
Training file consists of
image_id: Image Id which is also image file name
diagnosis: Categorical feature which rates the severity of blindness on a scale of 0 to 4
Performance Metric: Quadratic weighted Kappa
Quadratic weighted Kappa is a statistic that measures inter-annotator agreement. It is a score that lies between -1 and 1 that expresses the level of agreement between two annotators on a classification problem. 1 means complete agreement and -1 means complete disagreement.
- Importing Data
- Exploratory Data Analysis
- Exploring 2019 Data
- Exploring 2015 Data
- Exploring 2015 and 2019 Data combined
- Data Preprocessing
- Training Models
Importing Data2019 Data Source :https://www.kaggle.com/c/aptos2019-blindness-detection/data
There are 3662 Datapoints and each Datapoint represents Image file name and its diagnosis report examined by an Eye Specialist. File path is added manually into dataset to fetch Image from disk.
2015 Data Source : https://www.kaggle.com/c/diabetic-retinopathy-detection/data
There are 35,126 Datapoints and each Datapoint represents Image file name, its diagnosis report examined by an Eye Specialist and file path of Image in disk.
There are total of 38,788 Datapoints 2015 and 2019 data combined.
Exploratory Data Analysis
Exploring 2019 Data
level 1 blindness has 370 datapoints i.e, it occupies 10% of data
level 2 blindness has 999 datapoints i.e, it occupies 27% of data
level 3 blindness has 193 datapoints i.e, it occupies 5% of data
level 4 blindness has 295 datapoints i.e, it occupies 8% of data
weightage of class 0 : 0.4057617728531856
weightage of class 1 : 1.9794594594594594
weightage of class 2 : 0.7331331331331331
weightage of class 3 : 3.7948186528497407
weightage of class 4 : 2.4827118644067796
Exploring 2015 Data
level 1 blindness has 2443 datapoints i.e, it occupies 7% of data
level 2 blindness has 5292 datapoints i.e, it occupies 15% of data
level 3 blindness has 873 datapoints i.e, it occupies 2% of data
level 4 blindness has 708 datapoints i.e, it occupies 2% of data
weightage of class 0 : 0.27218907400232467
weightage of class 1 : 2.875644699140401
weightage of class 2 : 1.3275132275132275
weightage of class 3 : 8.047193585337915
weightage of class 4 : 9.922598870056497
Exploring 2015 and 2019 Data combinedThere are overall 38,788 Datapoints and each Datapoint represents Image file name as its diagnosis report examined by and Eye Specialist
level 2 blindness has 6291 datapoints i.e, it occupies 16% of data
level 1 blindness has 2813 datapoints i.e, it occupies 7% of data
level 3 blindness has 1066 datapoints i.e, it occupies 3% of data
level 4 blindness has 1003 datapoints i.e, it occupies 3% of data
weightage of class 0 : 0.2809197899692196
weightage of class 1 : 2.757767507998578
weightage of class 2 : 1.2331266889206804
weightage of class 3 : 7.277298311444653
weightage of class 4 : 7.734396809571286
Dataset OverviewBelow are the Images which are provided for Diabetic Retinopathy to detect the severity of blindness. Each row in the image is sorted from severity 0-4.
Cropping unwanted dataAs observed from above images, most of the images contain a black and gray backgrounds which is not essential and occupies extra space with unnecessary data. Here we are cropping down the images by removing the extra space and fit the subject exactly within frame, so that it will be easier to train model by focus on main subjects in the image.
Preprocess ImagesLike any real-world data set, it is common to encounter noise in both the images and labels. Images may contain artifacts, be out of focus, underexposed, overexposed, or different colour grading. The images were gathered from multiple clinics using a variety of cameras over an extended period of time, and also considering Human Errors will introduce further variation.
To reduce the effect of noise and enhance the images to clearly observe the abnormalities which shows the cause blindness, we are cropping the images into circular shape and using Ben Graham’s method to see abnormalities in more noticeable way.
The above cropped Images are preprocessed as below
Vectorization and Training Models
Train Test SplitData is split based on filepath of Images. 10% of Total Data is split into test data and 10% of remaining data is considered as Cross Validation data.
Vectorizing Class LabelsClinician has diagnosed and rated each image for the severity of DR on a scale of 0 to 4, which is a Multi-Class Problem with 5 classes. The class labels are Vectorized using One-Hot Encoding using sklearn.
Loading Image data for Training2015 and 2019 data combinedly have more than 38k images which is a lot of data to load in memory to process. tensorflow.data helps to load the large dataset directly from disk to process the data, without loading into memory.
Here a function is declared for preprocessing of data by converting the image into jpeg before training as soon as the data is loaded.
Tensorflow data loads Image data directly from given filepath and preprocess from the above function it as soon as it is loaded for training.
Training ModelsA Fully Convolutional Network (FCN) Model is built using Keras Subclassing. Since Keras utilizes object-oriented programming, we can actually subclass and then insert our architecture definition.
Model subclassing is fully-customizable and enables you to implement your own custom forward-pass of the model, which enables us to design Exotic architectures.
Below FCN model architecture is designed using Model Subclassing.
Keras Subclass Models can be built while fitting the training data i.e, fit fuction or it can also be built using build function as below.
Tensorboard LogTensorboard Graph of Kappa score on training above model for 300 epochs of both train and validation data is shown below.
- On manual checking of Image data it is observed that few Images are Out of Focus. RetinaNet is one the best Models available to overcome the problem.
- The Image data is preprocessed to 128*128 Images, we can also preprocess the images to higher resolutions like 256*256, 512*512 or higher, which could help on gainining more accurate data, Hence leading to higher Kappa scores.
- Improve Interpretation by Image Segmentation or Object Detection, which will help Doctors to show the anomolies in the retina which interprets the reason of prediction of Blindness, so that even if the predictions are wrong the doctors can double check in much detailed manner by focusing on predicted areas. This could help to improve the prediction as well as interpretation of model by adding more accurate data.
ConclusionMillions of people suffer from diabetic retinopathy and examining Detecting DR is a time consuming process if done manually, building a Neural Net model could reduce time taken examining the level of blindness on top of that due to portability of model this could increase the scale of examination at much lower efforts and costs. This could help millions of people to get an appropriate treatment before its too late.
The lesson I learnt is to explore and be updated so that we can apply suitable models for suitable situations. Imagine yourself into that problem, so that it will be easier to make assumptions, analogies, situations etc. Considering Heathcare Industry finding a suitable model is not enough it is also important to focus on efficiency like train time, performance and interpretability, because it is the question of ones life.