Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

centralize logic for file2asset(path, ds) #692

Closed
yarikoptic opened this issue Jun 30, 2021 · 2 comments · Fixed by #693
Closed

centralize logic for file2asset(path, ds) #692

yarikoptic opened this issue Jun 30, 2021 · 2 comments · Fixed by #693
Assignees

Comments

@yarikoptic
Copy link
Member

ATM upload does

            yield {"status": "extracting metadata"}
            try:
                asset_metadata = nwb2asset(
                    path, digest=file_etag, digest_type="dandi_etag"
                )
            except Exception as exc:
                lgr.exception("Failed to extract metadata from %s", path)
                if allow_any_path:
                    yield {"status": "failed to extract metadata"}
                    asset_metadata = get_default_metadata(
                        path, digest=file_etag, digest_type="dandi_etag"
                    )
                else:
                    yield skip_file("failed to extract metadata: %s" % str(exc))
                    return
            metadata = asset_metadata.json_dict()
            metadata["path"] = str(relpath)

so besides fall-back to get_default_metadata it adds a logic to convert to json_dict (could be omitted) but also fixes up the path to be relative to the dataset. For the purpose of metadata migration/updates etc, would be useful to centralize that logic so we do not need to (possibly incompletely) duplicate it within backups2datalad.py etc.

Could be a helper function or a class method on Asset, e.g. from_file then (IIRC has idea about dandiset) may be even ds does not need to be provided

@jwodder
Copy link
Member

jwodder commented Jun 30, 2021

@yarikoptic Should this function take an allow_any_path argument, or should it always fall back to get_default_metadata() on failure?

@yarikoptic
Copy link
Member Author

my current thinking is: For now I think it could just be a transfer of this logic. We will need to RF it later anyways to accommodate BIDS etc.

yarikoptic added a commit that referenced this issue Jul 6, 2021
Add get_asset_metadata() function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants