cleanup download utils

[pmeier]

Follow-up to #8237. We only had the _save_response_content as a separate function because it was used for the GDrive as well. Now, we can just inline it.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8273

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit dabc313 with merge base f21c6bd ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• CMake / windows (windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

Thanks @pmeier , LGTM but please check my comment below

[NicolasHug]

The new changes are only equivalent to the old ones if this continue was meant to be a break. Was this meant to be a break? In other words, can there be an empty chunk and then a non-empty chunk?

[pmeier]

TL;DR they are equivalent since content now can only be one type or iterator for which the branch will never trigger.

These keep-alive chunks are b"". Content iterators coming from requests could actually contain them. However, the content iterator coming from our _urlretrieve function cannot.

The way iter works when you have two inputs is quite unintuitive. Basically waht iter(lambda: response.read(chunk_size), b"") does is it calls the lambda forever until that returns the sentinel b"". If that happens it raises a StopIteration. Meaning

for chunk in iter(lambda: response.read(chunk_size), b""):
    if not chunk:
        # this will never be reached

[pmeier]

To answer your question directly:

In other words, can there be an empty chunk and then a non-empty chunk?

No.

items = [b"foo", b"", b"bar"]
idx = 0


def fn():
    global idx

    item = items[idx]
    idx += 1
    return item


for chunk in iter(fn, b""):
    print(chunk)

b'foo'