cellfie (CELL FI nd E r) automatically segments neurons in calcium imaging videos using fully convolutional neural networks.
Neuroscientists use calcium imaging to monitor the activity of large populations of neurons in awake, behaving animals (like in this beautiful example). However, calcium imaging can be very noisy, making neuron identification challenging. cellfie addresses this problem using a two stage convolutional neural network approach. First, a region proposal network identifies potential neurons. Next, an instance segmentation network iteratively identifies individual neurons.
cellfie is under active development. I'm collaborating with Eftychios Pnevmatikakis at the Simon's Foundation to see if neural networks combined with matrix factorization techniques (as used by CaImAn) outperforms current approaches to calcium imaging segmentation. The datasets used to train cellfie were meticulously created by Eftychios and his team.
First, a region proposal network segments all neurons from the background. Rather than passing enormous videos into the network, three summary images are created that collapse videos across time:
- correlation images that represent the correlation of each pixel to all neighboring pixels. The idea here is that pixels within a neuron will be correlated with neighboring within-cell pixels.
- standard deviation images that capture the variability of each pixel. Pixels belonging to neurons should have higher variability due to fluctuations in neural activity.
- median images that simply capture the median pixel value across time..
I use a U-Net-style fully convolutional neural network to turn these summary images into a heatmap where each pixel represents the probability that the pixel belongs to a neuron (depicted above).
But how can we find individual neurons? I train a second instance segmentation network that takes small subframes within the summary images as input. This network outputs a segmentation of the neuron centered within the current subframe (omitting other neurons in the frame!), in addition to a likelihood that there is a neuron centered at that location.
Data regularities are your friend! Many machine learning algorithms are built to handle data 'in the wild', such as images from cell phones or self-driving cars that are enormously variable. For example, Mask R-CNN uses an array of bounding boxes to make sure it can handle objects of different sizes. Scientific datasets, however, have remarkable stereotypy that can be leveraged when building new algorithms. In microscopy data, for example, we already know how big a cell should be! This makes our job way easier, and our algorithms way faster. Similarly, cellfie uses pretty shallow networks that are trained from scratch (without needing transfer learning) in only ~ a half hour (on a crummy GPU!), which would not be possible on standard datasets.