Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle subprocess disallowing preexec during shutdown #4879

Merged
merged 4 commits into from
Jul 13, 2023

Conversation

AdamWill
Copy link
Contributor

@AdamWill AdamWill commented Jul 5, 2023

In Python 3.12, you cannot pass a preexec_fn to subprocess.Popen during interpreter shutdown:

python/cpython#104826

This avoids the problem by giving startProgram an arg to not use a preexec_fn, and passing that all the way from anaconda's exitHandler through execWithRedirect and _run_program.

@github-actions github-actions bot added the f39 label Jul 5, 2023
In Python 3.12, you cannot pass a preexec_fn to subprocess.Popen
during interpreter shutdown:

python/cpython#104826

This avoids the problem by giving startProgram an arg to not
use a preexec_fn, and passing that all the way from anaconda's
exitHandler through execWithRedirect and _run_program.

Signed-off-by: Adam Williamson <awilliam@redhat.com>
@AdamWill
Copy link
Contributor Author

AdamWill commented Jul 5, 2023

This works OK in openQA testing.

A possible alternative is just to have the exit handler call subprocess directly; I don't think all these wrappers around it are doing much of any use on this path. But that seemed a slightly bigger change than this, so I went with this to start with.

@AdamWill
Copy link
Contributor Author

AdamWill commented Jul 6, 2023

This (or a different fix) must be merged before the next release, because Python 3.12 has been merged into Rawhide. If a new release is made without this bug fixed, it'll be built in Rawhide by packit, and that build will be broken. (Fortunately, now gating is enabled, it should be blocked by gating rather than getting in and breaking the compose).

Also, today's compose testing indicates this needs extending to the VNC shutdown code:

https://openqa.fedoraproject.org/tests/2005161#step/_boot_to_anaconda/5

I'm on vacation without a laptop, so if someone else can do that, it'd be great.

@VladimirSlavik VladimirSlavik self-assigned this Jul 12, 2023
@VladimirSlavik
Copy link
Contributor

Just a note, I thought of the following alternatives constrained to changes only in startProgram:

  • Check dynamically if preexec should happen based on sys.is_finalizing() . This appears as a silent side effect, so not really desirable.
  • Check if preexec is really needed - the function body is all conditionals - by testing all the clauses. Unfortunately this does not solve the problem, because reset_handlers is always True in the call path from execWithRedirect.
  • Propagate the reset_handlers flag instead of do_preexec - mostly the same problem as above, does not really explain that it's preexec that should not happen.

So I think this is the only way.

@VladimirSlavik
Copy link
Contributor

Needs:

  • fix the VNC code path above too
  • tests

Working on that.

@VladimirSlavik
Copy link
Contributor

/kickstart-test --testtype smoke

@AdamWill
Copy link
Contributor Author

Thanks! What did you think of the 'just have the exit handlers call subprocess directly' idea?

@VladimirSlavik
Copy link
Contributor

Ah, I missed that one. I think this way is better - we have all the logging and such in place for these calls, and will likely expect the logs to look the same for this.

I also feel that worrying too much about this might be premature optimization. The exit-ing code paths need modularization, soonish, and the exec* family of functions is also one of the areas that are known to need rewriting for contemporary Python. So this might not live too long anyway...

Copy link
Member

@jkonecny12 jkonecny12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants