Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Standalone CLI now estimates fieldmaps #408

Merged
merged 16 commits into from
Dec 6, 2023

Conversation

oesteban
Copy link
Member

The previous behavior (just "finding" estimators) is maintained with the argument --dry-run.

Copy link

codecov bot commented Nov 28, 2023

Codecov Report

Attention: 219 lines in your changes are missing coverage. Please review.

Comparison is base (53482c1) 80.59% compared to head (a77775f) 76.66%.

❗ Current head a77775f differs from pull request most recent head a682abe. Consider uploading reports for the commit a682abe to get more accurate results

Files Patch % Lines
sdcflows/config.py 73.83% 72 Missing and 12 partials ⚠️
sdcflows/cli/main.py 40.00% 47 Missing and 4 partials ⚠️
sdcflows/cli/parser.py 64.00% 30 Missing and 6 partials ⚠️
sdcflows/workflows/fit/base.py 8.33% 22 Missing ⚠️
sdcflows/cli/workflow.py 13.33% 13 Missing ⚠️
sdcflows/utils/telemetry.py 37.50% 10 Missing ⚠️
sdcflows/fieldmaps.py 71.42% 0 Missing and 2 partials ⚠️
sdcflows/_warnings.py 92.30% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #408      +/-   ##
==========================================
- Coverage   80.59%   76.66%   -3.93%     
==========================================
  Files          26       32       +6     
  Lines        2293     2824     +531     
  Branches      291      373      +82     
==========================================
+ Hits         1848     2165     +317     
- Misses        401      593     +192     
- Partials       44       66      +22     
Flag Coverage Δ
unittests ∅ <ø> (∅)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@oesteban oesteban force-pushed the enh/improve-standalone-cli branch 2 times, most recently from 216d148 to df93c7b Compare November 28, 2023 16:52
@oesteban
Copy link
Member Author

Well, at df93c7b, this is surprisingly executing the estimation of GRE fieldmaps. EPI fieldmaps failed in the robust average, so I'll need to check what went wrong.

@oesteban oesteban marked this pull request as ready for review December 4, 2023 15:23
docs/_static/config-example.toml Show resolved Hide resolved
# workflow construction jailed within a process preempts excessive VM buildup.
from multiprocessing import Manager, Process

with Manager() as mgr:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it needed to separate this, if you've already initialized the process pool?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could send it to the pool, that'd be fine. However, it's working this way in MRIQC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point was the pool is initialized, so isn't it safe to do this in the main process at this point? These managers tend to make debugging trickier, so if we can avoid it, I'd rather.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes, in theory we could use the main process without problems, indeed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I was removing this context to build the workflow in the main thread, and it could be interesting to keep it this way, as the workflow is created while OMP_NUM_THREADS is set to 1. This saves great numpy buildup.

We could still move it to the main process, but then it would make sense to make sure numpy is imported with OMP_NUM_THREADS = 1 (and that is then overridden in subprocesses).

sdcflows/cli/main.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/cli/parser.py Show resolved Hide resolved
sdcflows/cli/parser.py Outdated Show resolved Hide resolved
sdcflows/config.py Outdated Show resolved Hide resolved
Co-authored-by: Chris Markiewicz <markiewicz@stanford.edu>
Copy link
Member

@effigies effigies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go ahead and merge this when you're ready. It would be nice if we could get away from the Manager() indirection for building workflows eventually, but I don't want to hold this up on that.

@oesteban oesteban merged commit f8f35d7 into master Dec 6, 2023
24 checks passed
@oesteban oesteban deleted the enh/improve-standalone-cli branch December 6, 2023 14:10
@effigies effigies mentioned this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants