Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deal with multiple tableschemas in the same dataset #11

Open
jen-thomas opened this issue Mar 8, 2021 · 1 comment
Open

Deal with multiple tableschemas in the same dataset #11

jen-thomas opened this issue Mar 8, 2021 · 1 comment

Comments

@jen-thomas
Copy link
Contributor

jen-thomas commented Mar 8, 2021

Some datasets contain datasets with multiple data files that may have different formats and therefore require different tableschemas to describe them.

We need to be able to handle multiple tableschemas within the same dataset, e.g. see the branch dev-wip-1494924 which contains the dataset 10.5281_zenodo.1494923.

@MarcoAlbaETT
Copy link
Collaborator

After trying to ingest the 10.5281_zenodo.1494923 dataset I've found a "little" problem on the table schema.
In order to load the datapackage.json with the frictionless package I need to download all the table schema files used by the datapackage, and at the moment is not straight forward.

I've found 3 solutions:

  1. parse the datapackage.json file, find all the schema file names and download them
  2. download the repository zip file
  3. always create a single zip file with the datapackage.json and all the needed table schema files  

Solution number 3 need some work on your side but, thinking about what we discussed this morning (one zip with json files and one with data files), it might be the right way to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants