config: Add a media type for the configuration format

[wking]

So we can put these in image-spec CAS, or serve them over HTTP, or whatever.

[cyphar]

The reason this was not included from the start is because it breaks compatibility with Docker's config blobs (resulting in the manifest blobs referencing different blob IDs and so on).

[wking]

On Sat, Nov 05, 2016 at 09:56:04PM -0700, Aleksa Sarai wrote:

The reason this was not included from the start is because it breaks
compatibility with Docker's config blobs (resulting in the manifest
blobs referencing different blob IDs and so on).

Having a media type does not mean you have to use it in place of the
current image-spec config, so I think this would be a useful addition
even without an image-spec. And I think we can allow it in
image-spec without breaking Docker compat; details on my dual-stack
plan in opencontainers/image-spec#451.

[cyphar]

Ah, sorry I thought this PR was against image-spec. 😉

[mrunalp]

@vbatts @philips PTAL

[crosbymichael]

How and when is this used for the runtime-spec. I'm confused why we need it here?

[cyphar]

@crosbymichael I think that @wking is implying that he would like to have config.json stored inside OCI images. I think this is not a good idea for several reasons, mainly because IMO image configuration has a very different set of requirements to runtime configuration.

Image configuration is more declarative, while runtime configuration is much more fine-grained and descriptive. Things being "open to interpretation" is a benefit here IMO because runtimes might have different opinions to the image author about how the image will be used.

As for the actual issue of having a mediatype in config.json I'm not really convinced why it would be necessary. It makes sense for the image CAS, but I don't see why it makes sense for the runtime spec.

[wking]

On Tue, Nov 15, 2016 at 02:54:47AM -0800, Aleksa Sarai wrote:

I think that @wking is implying that he would like to have
config.json stored inside OCI images.

This is one use case (covered in more detail in
opencontainers/image-spec#451). You also use media types in a number
of other situations (e.g. validation, serving files over HTTP,
attaching files in email, …). Having a canonical media type for the
runtime config format helps in all of those cases.

As for the actual issue of having a mediatype in config.json…

I'm not suggesting a mediaType JSON property (like image-spec uses
1). I'm just suggesting we give the runtime config format a media
type.

[crosbymichael]

I see. like @cyphar said, this is a really bad idea and that is the reason these two are separate. The runtime holds host specific information and the right way to handle the runtime spec is that it is generated at runtime and destroyed along with the container. This spec should not be stored or used inside the image.

[wking]

On Wed, Nov 16, 2016 at 10:00AM -0800, Michael Crosby wrote:

like @cyphar said, this is a really bad idea and that is the reason these two are separate

This is presumably in response to opencontainers/image-spec#451. All this PR is doing is giving the configuration a media type. I don't see why that's a bad idea.

The runtime holds host specific information and the right way to handle the runtime spec is that it is generated at runtime and destroyed along with the container. This spec should not be stored or used inside the image.

I'm not convinced, but rejecting opencontainers/image-spec#451 is a good way to say “the runtime spec should not be stored in the image”. Rejecting this PR, on the other hand, is saying “we don't want a machine-recognizable name this schema, because we think having such a name is a really bad idea”. I don't see why that would be the case.

For example, it would be helpful to tell a validator “I'd like you to validate this application/vnd.oci.runtime.config.v1+json”, because then you don't need per-schema validation endpoints for the config and (say) the state JSON.

[dqminh]

@wking i think it would be nice to list out the cases where this is helpful/necessary, because "put these in image-spec CAS, or serve them over HTTP," is definitely what we dont want to do, which probably leads to the confusions here.

[wking]

On Wed, Nov 16, 2016 at 12:47:49PM -0800, Daniel, Dao Quang Minh wrote:

@wking i think it would be nice to list out the cases where this is
helpful/necessary…

Necessary is too high a bar. Not having a canonical media type just
means that external consumers trying to identify the type need to coin
their own name. Defining a canonical name here just makes interop
easier by avoiding fragmentation based on 3rd party choices. With
that in mind, here are some cases where I think a canonical
runtime-config media type would be helpful:

Validation

We briefly had a validator.opencontainers.org repository (e.g. see
opencontainers/image-spec#203), although it seems to be gone now. If
we look at other validator examples, 1 has a few validation options
which all include a “Document Type” dropdown. They use strings
(e.g. “XHTML 1.1”), but that may be because they have several versions
packed into a single media type (e.g. application/xhtml+xml 2),
while the example set by image-spec is to use versioned media types
3. If we had media types for the runtime-spec JSON, we could have a
single web validator which could validate all OCI content.

Flexible runtimes

With a runtime that supports multiple configurations, a media type
provides a convenient, unambiguous way to switch between configuration
handlers.

$ flexibleRuntime --type application/vnd.oci.runtime.config.v1+json --config config.json create hello-1
$ flexibleRuntime --type application/vnd.lxc.runtime.config --config lxc.container.conf create hello-2

Those are command-line examples, but you could also imagine a runtime
that received requests via HTTP:

$ curl -XPOST -H 'Content-Type: application/vnd.oci.runtime.config.v1+json; charset=utf-8' -d - https://container.example.com/create/hello-3 <config.json

Emailing configurations

Asking for help:

MIME-Version: 1.0
Content-Type: application/vnd.oci.runtime.config.v1+json; charset=utf-8
Content-Transfer-Encoding: base64
Content-Disposition: inline
To: support@example.com
Subject: Why are these mounts not working?
…

{
"ociVersion": "1.0.0-rc2",
…
}

… because "put these in image-spec CAS, or serve them over HTTP," is
definitely what we dont want to do, which probably leads to the
confusions here.

I'm still not convinced, but that discussion is part of
opencontainers/image-spec#451. My impression of the resistance to
this PR is that folks are concerned about the image-spec implications.
Maybe it would be more productive to try to hash that out in
opencontainers/image-spec#451 and then come back to this PR once
opencontainers/image-spec#451 reaches a consensus?

[mikebrow]

The media type for this configuration file is [`application/vnd.oci.runtime.config.v1+json`](application/vnd.oci.runtime.config.v1+json)

[wking]

What are you suggesting? I don't think that Markdown link will resolve. Are you suggesting I drop the JSON and ociVersion requirements for that media type? Do you expect those requirements to be overly restrictive? How do you expect consumers to process instances of this media type if we break either restriction?

[mikebrow]

I'm suggesting the text for this file is configuration is includes a typo. And was hoping for a link to something that describes the media types. Should've looked it up, pretty sure I saw it somewhere around here :)

[wking]

I'm suggesting the text for this file is configuration is includes a typo.

Ah, right. Fixed with 8696dfa → 32a192f, although I've gone with “format” instead of “file”.

And was hoping for a link to something that describes the media types. Should've looked it up, pretty sure I saw it somewhere around here :)

You may be thinking of this. I'm not aware of anything similar in this repo.

[mikebrow]

Looks good to me.

[RobDolinMS]

@stevvooe PTAL

[stevvooe]

@wking So, I don't see a problem with defining a media type here. We should actually make these declaration in a central location (currently the image spec).

For runtime config, we should include a warning about the utility and dangers of distributing them. These might be two separate PRs. One to runtime repo with warning on distribution and one to image-spec registering the mediatype.

@philips Does this sound reasonable?

[wking]

On Wed, Jan 11, 2017 at 02:40:19PM -0800, Stephen Day wrote: We should actually make these declaration in a central location (currently the image spec).

I think it's easier to make the fundamental claim in the spec that declares the format. Having a central location that collects any registered media types makes sense, but the IANA already does that [1] (and registration for the vendor tree is optional [2]). If we want our own OCI-wide collection of declared media types, I think that should probably be the responsibility of whoever's maintaining www.opencontainers.org (which may move back into a gh-pages site at some point [3]).

For runtime config, we should include a warning about the utility and dangers of distributing them.

I'm fine with this, but (as opencontainer/image-spec#451 shows ;), I don't have a clear handle on the dangers. I'll leave that warning PR for someone who does. [1]: https://www.iana.org/assignments/media-types/media-types.xhtml [2]: https://tools.ietf.org/html/rfc6838#section-3.2 [3]: opencontainers/web#20 (comment)

[crosbymichael]

-1 because I don't what to have anything in this format that promotes transfer or distribution, which was the original reason for this PR.

I don't think it belongs in the runtime-sepc

[mikebrow]

Maybe change it to say that the runtime-spec does not include an image specification for distribution. Then provide a link to the OCI image specification as one possible future image distribution spec?

Or should we just be silent to the other project here?

[wking]

-1 because I don't what to have anything in this format that promotes transfer or distribution, which was the original reason for this PR.

“transfer or distribution” are hard to pin down. The validation and flexible runtime cases from this comment may be about transfer from one local process to another. Do we not want to facilitate those?

Maybe change it to say that the runtime-spec does not include an image specification for distribution.

I'm fine with this. Maybe wording like:

This specification does not cover distribution (which is covered in the OCI Image Format Specification).

or did you have something else in mind?

[vbatts]

the only value i see in this is having some kind of versioning of the docs being referenced. That could be handled by a dedicated field for document version. By giving these docs a media-type gives it encouragement to be distributed, which is not wanted.
-1 on this.

[crosbymichael]

Many of the maintainers on both image and runtime are against this universally as this is not inline with the overall goals of the project and have security implications if encouraged to distribute the runtime configuration.

If you want to store and distribute these then go ahead, but we don't want to add anything to encourage this.