graph cut for image segmentation

[weilinear]

This is an early pull request for image segmentation using graph cut. I adopt code spectral_embedding and discretize from scikit-learn with and remove those unused features within the code. I add an example to plot_segmentations based on the result from slic:

The function calculates the affinity matrix as in random_walker_segmentation and then apply the graph cuts algorithms to get the pixel-wise discrete labels. Also, it can take pre-segmentation to accelerate the process. Image that with more than 1000 pixels wll suffer from the svd on large laplacian matrix. It currently supports callable function, numpy ndarray and some predefined methods(currently only slic) as a preprocessing method.

It may take some time to clean the codes and repair some trivial mistakes before review. Just send the pull request to make sure it is in the correct direction :)

PS: Actually I am not sure whether I can use code from scikit-learn? @amueller

[amueller]

That is pretty cool :)
There is a paper on this method, though I forgot the name. I'll have a closer look on how you compute the affinity matrix later and maybe I can dig up the reference.

As both projects are BSD-Licensed, moving code is no problem. You just have to leave in the authors afaik.
The question is whether the code-duplication is worth it :)

Btw, I wouldn't call this "graph cuts" but rather "normalized cuts".
I think "graph cuts" is usually reserved for methods that do a max-flow / min-cut.

[weilinear]

@amueller Thanks for the information. I will ping you after I clean the code, as it is quite premature for now (it also fails on travis currently). Directly copy the code may not be a good idea as spectral_clustering in sklearn does many checks that is not quite useful for segmentation. I think @GaelVaroquaux has done a lot in making the spectral clustering robust and so use it may save a lot of time to build upon code from sklearn.

Comments are welcome.

[stefanv]

/ping @kuantkid Any further progress?

[weilinear]

@stefanv I am busy job hunting and thesis those months. I will return back after two weeks or so. Sorry for the lag :). If anyone is interested, please go ahead.

[stefanv]

@kuantkid All the best! We look forward to having you back.

[thouis]

You might want to implement the more robust scheme from this paper (see section 6): http://berkeley.intel-research.net/arahimi/c/papers/rahimi-ncut.pdf

[stefanv]

/ping @kuantkid

[weilinear]

@stefanv Sorry this is terribly delayed by various personal reasons during this graduate time. I will begin work now :)

[stefanv]

@kuantkid Are you planning on working on this PR again?

[souravsingh]

Is the PR still being worked? If not, I can pick from where the work was left and attempt to finish up.

[Borda]

btw, what about using this segmentation? https://github.com/Borda/pyImSegm

[soupault]

@Borda great work! How fast is your code?

[Borda]

@soupault it depends on several parameters/aspects, such as how many features you want to compute, what is the size of superpixels, etc., but in general, it is a couple of seconds...

[soupault]

@Borda could you, please, elaborate on what you have in mind? I see that your repo implements, so to say, a framework for segmentation. Are you suggesting to consider your implementation or Graph Cut, or something more?

[Borda]

@soupault I am not sure if I should suggest anything, I was giving an example that there are also some other graph-cut based segmentation... I think that using super-pixels features is quite useful and make the segmentation more robust and running graph-cut on large images is impossible to compare to running it on an image of the same size decomposed into super-pixels... The graph-cut is standard, and computed features are simple or what should I elaborate?

[alexdesiqueira]

Hi @souravsingh @Borda are any of you still interested in bringing this to life again? :)

[Borda]

it looks quite old... @alexdesiqueira do you mean to finish this PR or what I proposed makes Graph-Cut on superpixels? I could do the superpixel version, just need to find python graph-cut running on all OS, because pyGCO is now compiled only for Linux... I was considering Borda/pyImSegm#21

[alexdesiqueira]

Hey @Borda,
I checked your issue (Borda#21) and it seems a nice algorithm! The papers where it is based ([1, 2] and others) seem relevant, so I'm in favor of having it here.
@scikit-image/core are we in favor of closing this PR and @Borda opening a new one on graph-cut on superpixels?

[jni]

@alexdesiqueira @Borda the superpixel version exists in skimage.future.graph. We definitely need to pull that out of there... 🤦‍♂

[jni]

See https://scikit-image.org/docs/dev/auto_examples/segmentation/plot_ncut.html#sphx-glr-auto-examples-segmentation-plot-ncut-py

[Borda]

@jni so it means that there is no need for another superpixel - graphCut, right?

[jni]

That's my reading of the situation, yes... This PR actually predates skimage.future, but, as many do, it got stuck in limbo while that was added. =\

[alexdesiqueira]

I see two ways we could proceed here:

1. Maybe it's time to discuss putting the graph_cut algorithms (normalized, threshold, n cut) at skimage.graph, changing API, all the stuff you would like to do, @jni. @Borda would you like to take care of that?
2. I checked the papers, and it seems that the algorithms differ (the one @Borda suggesting being "newer"). Since we have some cut algorithms already, I wouldn't mind us having another algorithm inside (maybe in skimage.graph.cut). What do you think about that @jni?

[alexdesiqueira]

@alexdesiqueira @Borda the superpixel version exists in skimage.future.graph. We definitely need to pull that out of there...\

As per @jni's comment, I'm closing this PR. The plan is to bring the respective function in future.graph to graph.
Thank you everyone for your work on this one!