Split make_SILVA_132_16S_classifier into sequence extraction and training?

[erikrikarddaniel]

Sequence extraction takes forever on the UPPMAX cluster (4-5 days), and uses little memory, whereas the training step requires more memory (20-25 GiB) and takes much shorter (< 3h). To make it possible to set better cpu and memory limits for this, I would therefore suggest that we split this process into two: The first would run the currently first four steps (unzipping, qiime imports and read extraction) and the second would just be the training step.

[d4straub]

Is this still the case (1.1.2/dev)? The whole process make_SILVA_132_16S_classifier takes <3h for me.

[erikrikarddaniel]

Den ons 28 okt. 2020 17:03Daniel Strau <notifications@github.com> skrev:

Is this still the case (1.1.2/dev)? The whole process make_SILVA_132_16S_classifier takes <3h for me.

I should think so, but I haven't tried since I've been running a ready made classifier for a long time now. In any case, I can't see that it would hurt to make two processes. Emelie could do this. /D

…

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#147 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALTHQHQJZYAJSBQHHDCQO3SNA6GZANCNFSM4MZKGYDA> .

[d4straub]

Sure, I have no reservations against splitting the process.

[d4straub]

This was solved in the linked PR.