extension manifest

[pwinckles]

In PR #31 I proposed the idea of having a JSON file that lists the extensions that are used on an object or a storage root. I am creating this ticket for further discussing the idea. Later in the PR, I described an alternative approach where this manifest file also serves to store extension configuration instead of each extension having its own configuration file.

I proposed that this file live at extensions/extensions.json in the storage root and object extension directories. This knowingly contradicts existing language in the OCFL spec that forbids files under the extensions directory. If this considered immutable, a workaround could be to stick the manifest in a subdirectory.

When thinking about how OCFL extensions should work, there are some key questions:

1. How do you know when an extension is being used?
2. Can extensions reference other extensions, and, if so, how?
3. Can the same extensions be used multiple times on the same object/storage root?
4. What happens when an extension is no longer used or it's configuration changes? Is this something that should be preserved?
5. How common is it for extension configuration to change?
6. Is there a relationship between storage root extensions and object extensions? If so, what is its nature? Can the storage root contain extensions that affect objects?
7. Can different objects in the same repository use different extensions?
8. How easy is the extension system to use if you're using an object store?

Not all of those questions are necessarily relevant to whether or not there is an extension manifest/registry file, but I was on a roll writing them.

I think we could devise a reasonable system with or without the file, but, to me, it is much cleaner with the file. For example, let's say that we don't have a manifest and we want to keep old extension configuration. With those requirements, you could create extensions/enabled and extensions/historic (or whatever) directories. In the first, every extension that the object/storage root uses MUST have a directory named for its registered name that, at the minimum, MUST contain a configuration JSON file. This is true even if the extension has no parameters. Clients process the contents of this directory when an object/storage root is accessed, loading extensions by directory name.

The other directory is used to store old extension configuration in the same format as the contents of extensions/enabled. The only difference is that you'd have to come up with a scheme to dedup the directory names, timestamp or something, to account for the possibility that the directory may need to contain multiple different entries for the same extension.

It works, but, to me, the whole thing works more smoothly when there is a manifest. I think this is particularly true when it comes to extensions that are not parameterized (or are only using default parameters) and supporting object stores. Yes, I can fake an ls in S3 to extract the names of the extensions I need to load, but it's a heck of a lot more inconvenient than just downloading a file that tells me exactly what I need to know.

As mentioned earlier, the manifest approach could be implemented in a way that is more similar to the original directory based approach, or it could be taken a step further and encapsulate all of an object's (or storage root's) extension configuration. Instead of each extension having its own configuration file, there would only ever be at most one unified configuration file per object/storage root. There's an example of this here.

Personally, I find this single configuration file approach the easiest of the three to work with. But, I am curious how others weigh the pros and cons of each approach (and if there are any other ideas), and if there is a consensus on what is the best path forward.

[zimeon]

Separate from the substance of this issue, I think we should not consider calling anything other than the manifest in the object inventory a manifest -- that would cause unnecessary confusion.