Nuclei Classification and localization

Three class problem

We find and classify the normal epithelial, malignant epithelial and lymphocyte cells in subsections of H&E stained histology images taken at 20x magnification. Example with the malignant cells labelled in red:

Some visualizations of the data, as well as some problem areas are in exploring_data notebook.

Procedure: The general outline of our method is:

1. Classifier - Train a classifier to detect if an image is centered on a normal, malignant or lymphocyte nuclei, or else not on any nuclei. We create a data set by selecting square regions centered at the nuclei centers. We test different fully convolutional and regular CNNs. Augmentation is with flips, rotations and cropping. (code)
2. Heat Maps - Apply the classifier to the image, outputting a 4 dimensional heatmap of the probability of each cell class. (code)
3. Cell Locations - Based on the heat maps, output locations and classifications for the cells using non-maximum supression

Some of the testing is included in this notebook

---

Lymphocyte

• This model is also tested on a public lymphocyte data set in this notebook, but without any fine tuning it is not effective.
• We also test training it on the lymphocyte dataset with better results in this notebook

---

Conclustions

---

Other regression based models and unet based models are also tested, but were ineffective.