Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial auxiliary resources draft #156

Merged
merged 14 commits into from
May 6, 2020
Merged

Initial auxiliary resources draft #156

merged 14 commits into from
May 6, 2020

Conversation

justinwb
Copy link
Member

Included in this pull is a candidate proposal from the data interoperability panel for the handling of resource metadata in a Solid server. Specifically, this proposal introduces a mechanism for linking to metadata about resources in the Solid Ecosystem, and enumerates a set of reserved metadata types.

Remaining issues have been identified inline. Most noticeable will be the need to identify the contents of identified metadata types. The panel felt it was appropriate to raise those discussions to the editorial level first, as the various metadata types cross panel boundaries.

It should be noted that a significant amount of review and iteration was conducted over the course of several months, which was captured in solid/data-interoperability-panel#32.

@csarven
Copy link
Member

csarven commented Mar 23, 2020

Can we make sure to link and track each criteria in the proposed document to a corresponding issue (or remarks, questions raised within) or vice-versa?

@justinwb
Copy link
Member Author

Can we make sure to link and track each criteria in the proposed document to a corresponding issue (or remarks, questions raised within) or vice-versa?

@csarven are you talking about historical linkage, or creating new issues for any issues identified inline in the document? By citing solid/data-interoperability-panel#32 we capture just about all of the historic context. I can see reason already to make new issues for those raised inline in the submission which I can create today (unless I'm misinterpreting!)

@csarven
Copy link
Member

csarven commented Mar 23, 2020

In order to manage the issues in the solid/specification repository, we should track all criteria in the proposed document. See for example some of the commits in https://github.com/solid/specification/commits/editors-draft and how it references back to the issues and rough consensus that was reached in this repository. It helps to know the coverage and how to handle the open issues and their states in the project board going forward. Note that I'm not saying that all criteria in the proposed document has to have a corresponding issue in this repository - I know that not everything is covered here but was instead covered in the panel... and in that case, it should link to the issues / rough consensus that's documented in solid/data-interoperability-panel.

Copy link
Contributor

@RubenVerborgh RubenVerborgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tons of comments (that will take quite some time to process); main remark is to have separate server and client sections.

main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
@justinwb justinwb changed the title Initial resource metadata draft Initial auxiliary resources draft Apr 7, 2020
Copy link
Contributor

@RubenVerborgh RubenVerborgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments have been inlined as issues, so fine with me.

The Shape validation auxiliary resource directly associated with a given
resource
is discovered by the client via the `rel=http://www.w3.org/ns/solid/terms#shape`
`Link` relation in a `Link` header. Conversely, the resource being described by
Copy link
Member

@dmitrizagidulin dmitrizagidulin Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the history behind having two discovery mechanisms for shape? (shape and describes)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use this convention on all of the auxiliary resources that have a 1:1 relationship with the resource they have been associated with. Essentially its just a way to have a bi-directional association, specifically:

Resource A -> Shape Validation V
Shape Validation V -> Resource A

Note that the shape validation resource is not meant to actually link straight to the shape itself, but to an RDF document that will point to the shape(s), contain some other helpful metadata like type of shape(s), enforcement level - (warn, fail), etc.

Now that aux. resource section is wrapping I plan to start documenting that one, so there should be more detail soon.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the shape validation resource is not meant to actually link straight to the shape itself, but to an RDF document that will point to the shape(s), contain some other helpful metadata like type of shape(s), enforcement level - (warn, fail), etc.

Why isn't the shape resource self-describing ie. including both the shape and anything else that's relevant?

The name of the property signals "shape", the resource is called "shape resource".. but when the client gets there, they are directed to another resource which is the actual shape?

Too many meta layers for my taste.

We have rel=acl, describedby,.. targeting the most essential resource. How is the metadata (eg. enforcement level) about the shape resource (ie. the target of rel=shape) is of use to the client?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't the shape resource self-describing ie. including both the shape and anything else that's relevant?

The shape resource is public and doesn't know anything about the context in which you are applying it. For example, you may only want to warn when the validation fails. That a decision you make in how to apply the validation, not the author of the shape.

The name of the property signals "shape", the resource is called "shape resource"..

Agree that the property name is misleading - it might be best to change it to something else like "constraint" or "validation". Also there's nowhere in this document that the words "shape resource" are used?

but when the client gets there, they are directed to another resource which is the actual shape?

The purpose of this auxiliary resource is to illustrate one or more validation constraints set on that resource. It's not limited only to the shape itself, but also other parameters related to how that validation should be evaluated / applied.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this auxiliary resource is to illustrate one or more validation constraints set on that resource. It's not limited only to the shape itself, but also other parameters related to how that validation should be evaluated / applied.

That's fine but I don't see a reason why that's not all (including all constraints as well as any other related information) be stated in the auxiliary shape resource.

A resource representation whose shape is constrained by a given ShEx schema that links to an auxiliary resource defining that schema.

Sounds closer to what I'm saying than what you're saying but perhaps I'm misinterpreting.

In any case, I can't draw the underlying rationale from the text or arguments. I'm suspecting two reasons for having an intermediary (auxiliary) resource:

  • the lifecycle of the shape (constraints) information shouldn't depend on the lifecycle of the primary resource, so the auxiliary resource hop is a way to let the shapes live on. On deleting primary resource, auxiliary resource can get deleted without touching the shapes (which happen to be elsewhere and possibly used by other resources).

  • there are multiple approaches to shape validation (SHACL/ShEx) so instead of picking one to associate resources with, we have the intermediary auxiliary resource to point at them.

The first seems reasonable.

The second less so - after all, no particular reason why there can't be specific link relations point at shacl or shex resources. On that note, at least one of these things needs to be required in order to have minimum level of interop.

@dmitrizagidulin
Copy link
Member

+1 from me, this is a crucial and quite exciting addition to the spec.
(Also, I will definitely be bringing it up for discussion / consideration at the Secure Data Storage WG).

main/resource-access.bs Outdated Show resolved Hide resolved
Co-Authored-By: Aaron Coburn <acoburn@apache.org>
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
main/resource-access.bs Outdated Show resolved Hide resolved
@dmitrizagidulin
Copy link
Member

Minor request - would it be possible to have an example HEAD request that includes all of the auxiliary resource types in the response? Just so that developers can see it at a glance?

@justinwb
Copy link
Member Author

justinwb commented May 5, 2020

Minor request - would it be possible to have an example HEAD request that includes all of the auxiliary resource types in the response? Just so that developers can see it at a glance?

@dmitrizagidulin I agree this is useful. I wrote one, but then it felt like it would fit better in a corresponding primer document to the solid protocol spec. Thoughts?

@dmitrizagidulin
Copy link
Member

@justinwb oh, if you have one, please add it to this PR! We definitely need all the examples we can get our hands on, in the spec.

@csarven
Copy link
Member

csarven commented May 5, 2020

I agree. It is useful to give a brief example in this context. We can always edit later.

@justinwb
Copy link
Member Author

justinwb commented May 6, 2020

@justinwb oh, if you have one, please add it to this PR! We definitely need all the examples we can get our hands on, in the spec.

@dmitrizagidulin @csarven expanded the discovery examples to cover all types in this commit

@dmitrizagidulin
Copy link
Member

@justinwb Perfect, thanks!

Copy link
Member

@dmitrizagidulin dmitrizagidulin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 from me.

Copy link
Member

@csarven csarven left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't say I'm completely comfortable with the decision around auxiliary shape resource acting as a way to discover the actual shape validation (+ "metadata") resources but I'm okay to not block the PR for that either at this point - it is a design decision. I hope we can address the key concerns in: https://github.com/solid/specification/pull/156/files/93e92098f4c7dabb4bb62bdb917504550e65a313#r420914309 at some point. As I read it, how interop will happen around shapes is not clear cut. One client may use SHACL, the server may be okay with it, but another client may only understand ShEx.

@justinwb
Copy link
Member Author

justinwb commented May 6, 2020

I can't say I'm completely comfortable with the decision around auxiliary shape resource acting as a way to discover the actual shape validation (+ "metadata") resources but I'm okay to not block the PR for that either at this point - it is a design decision. I hope we can address the key concerns in: https://github.com/solid/specification/pull/156/files/93e92098f4c7dabb4bb62bdb917504550e65a313#r420914309 at some point. As I read it, how interop will happen around shapes is not clear cut. One client may use SHACL, the server may be okay with it, but another client may only understand ShEx.

Completely understand and agree. There is another section that @ericprud and i plan on contributing that will detail the specifics for shape validation. As part of authoring and reviewing that I'm confident we'll address those concerns.

@justinwb justinwb merged commit 7af6617 into master May 6, 2020
@csarven csarven deleted the resource-metadata branch June 11, 2020 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants