Error when using --preserve-tree option

[kabilar]

Hi @jwodder, I might be doing something wrong, but I am receiving an error with the --preserve-tree option.

When downloading a single asset from Dandiset 724 with the --preserve-tree option, the asset does not download and I receive a non-specific error that is not in the log file. Without the --preserve-tree option, the asset downloads successfully.

Thank you.

Command

dandi download --preserve-tree https://api.dandiarchive.org/api/assets/dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7/download/

Error

2024-08-05 15:59:18,154 [    INFO] Logs saved in /Users/kabilar/Library/Logs/dandi-cli/2024.08.05-20.59.17Z-57918.log
Error: 

Log file

2024-08-05T15:59:17-0500 [INFO    ] dandi 57918:8265534464 dandi v0.62.4+15.gf71ce95, hdmf v3.11.0, pynwb v2.5.0, h5py v3.10.0
2024-08-05T15:59:17-0500 [INFO    ] dandi 57918:8265534464 sys.argv = ['/Users/kabilar/miniconda3/envs/dandi/bin/dandi', 'download', '--preserve-tree', 'https://api.dandiarchive.org/api/assets/dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7/download/']
2024-08-05T15:59:17-0500 [INFO    ] dandi 57918:8265534464 os.getcwd() = /Users/kabilar/Desktop
2024-08-05T15:59:17-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 Starting new HTTPS connection (1): rig.mit.edu:443
2024-08-05T15:59:17-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 https://rig.mit.edu:443 "GET /et/projects/dandi/dandi-cli HTTP/1.1" 200 579
2024-08-05T15:59:17-0500 [DEBUG   ] dandi 57918:8265534464 Running a newer version (0.62.4+15.gf71ce95) of dandi/dandi-cli than available (0.62.4)
2024-08-05T15:59:17-0500 [DEBUG   ] dandi 57918:8265534464 Parsing url https://api.dandiarchive.org/api/assets/dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7/download/
2024-08-05T15:59:17-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 Starting new HTTPS connection (1): api.dandiarchive.org:443
2024-08-05T15:59:17-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 https://api.dandiarchive.org:443 "GET /api/info/ HTTP/1.1" 200 354
2024-08-05T15:59:17-0500 [DEBUG   ] dandi 57918:8265534464 Parsed into BaseAssetIDURL(instance=DandiInstance(name='dandi', gui='https://dandiarchive.org', api='https://api.dandiarchive.org/api'), dandiset_id=None, version_id=None, asset_id='dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7')
2024-08-05T15:59:17-0500 [DEBUG   ] dandi 57918:8265534464 GET https://api.dandiarchive.org/api/assets/dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7/info/
2024-08-05T15:59:17-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 Starting new HTTPS connection (1): api.dandiarchive.org:443
2024-08-05T15:59:18-0500 [DEBUG   ] urllib3.connectionpool 57918:8265534464 https://api.dandiarchive.org:443 "GET /api/assets/dd10fbc4-9ddf-4129-b6d6-412f7a07ffe7/info/ HTTP/1.1" 200 2071
2024-08-05T15:59:18-0500 [DEBUG   ] dandi 57918:8265534464 Response: 200
2024-08-05T15:59:18-0500 [DEBUG   ] dandi 57918:8265534464 Caught exception 
2024-08-05T15:59:18-0500 [INFO    ] dandi 57918:8265534464 Logs saved in /Users/kabilar/Library/Logs/dandi-cli/2024.08.05-20.59.17Z-57918.log

[kabilar]

cc @aaronkanzer

[kabilar]

Also, in the docs it says "Implies --download all". What does this mean? Thanks.

[jwodder]

@kabilar This should be fixed by #1476. However, note that, because you're downloading a URL that contains only an asset ID and not a Dandiset ID, the asset's Dandiset cannot be determined, and as a result no dandiset.yaml will be downloaded and no {dandiset_id}/ directory will be prefixed to the downloaded file.

Also, in the docs it says "Implies --download all". What does this mean? Thanks.

Genuine question: What do you think it means? Did you not notice the --download option in the same documentation?

[kabilar]

Thank you, @jwodder. Your fix resolves the error I received.

Also, in the docs it says "Implies --download all". What does this mean? Thanks.

Genuine question: What do you think it means? Did you not notice the --download option in the same documentation?

Perhaps I am misunderstanding the --download all option. I would assume that "Implies --download all" means that all assets in the Dandiset would be downloaded.

[jwodder]

@kabilar No, --download all means "Download both the assets and the dandiset.yaml" (as opposed to just downloading assets or just downloading the dandiset.yaml). --download is mainly intended for use when downloading a URL pointing to a Dandiset.

[kabilar]

Thank you, John. That makes sense.

[github-actions]

🚀 Issue was released in 0.63.0 🚀