Raise errors on all ranks for checkpoint download failures

[irenedea]

What does this PR do?

Raises errors on all ranks if one of the ranks experiences a download failures in order to address a known issue where checkpoint errors are not raised correctly if only not all ranks fail: pytorch/pytorch#122529

Manual Test Runs

Loading valid checkpoint still works: valid-checkpoint-sUUyo9
Loading corrupt checkpoint errors: corrupt-checkpoint-5RYyIa

What issue(s) does this change relate to?

Before submitting

• Have you read the contributor guidelines?
• Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
• Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
• Did you update any related docs and document your change?
• Did you update any related tests and add any new tests related to your change? (see testing)
• Did you run the tests locally to make sure they pass?
• Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

[mvpatel2000]

LGTM modulo error message -- can we be more descriptive on why we need to kill processes and what action user should take (view logs on failed ranks)

[bigning]

thanks for the nice fix!