proposals: add release-approval-process #15
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # OCI Project Release Approval Process v1.0 | ||
|
|
||
| OCI projects need a standard process for making releases so the community of maintainers can consistently know when something can be tagged and released. As the OCI maintains three categories of projects: specifications, applications, and conformance/testing tools we will set different rules for each. | ||
|
|
||
| ## Specifications | ||
|
|
||
| **Planning a release:** Every OCI specification project SHOULD hold a weekly meeting that involves maintainers reviewing pull requests, debating outstanding issues, and planning releases. This meeting MUST be advertised on the project README and MAY happen on a phone call, video conference, or on IRC. Maintainers MUST send updates to the dev@opencontainers.org with results of these meetings. Maintainers MAY change the meeting cadence once a specification has reached v1.0.0. The meeting cadence MUST NOT be greater than once every four weeks The release plans, corresponding milestones and estimated due dates MUST be published on GitHub (e.g. https://github.com/opencontainers/runtime-spec/milestones). | ||
|
|
||
| **Making a release:** OCI specification projects MUST announce intentions to release with two project maintainer sponsors (listed in the repo MAINTAINERS file) on the dev@opencontainers.org mailing list. After the announcement a two-thirds super majority of project maintainers MUST reply to the list with an LGTM within one week for the release to be approved. The maintainers MUST wait a full week for maintainers to reply but if all maintainers reply with an LGTM then the release MAY release earlier except in the case of a major releases. | ||
|
|
||
| **Rejecting a release:** A project maintainer MAY choose to reply with REJECT and MUST include a list of concerns filed as GitHub issues. The project maintainers SHOULD try to resolve the concerns and wait for the rejecting maintainer to change their opinion to LGTM. However, a release MAY continue with a single REJECT as long two-thirds of the project maintainers approved the release. | ||
|
There was a problem hiding this comment. So if a project with seven maintainers gets 5 LGTMs and 2 REJECTs, it's rejected? If so, I think you want to reference this rejection exception in the “Making a release” section. There was a problem hiding this comment. There are lot of problems with restricting this to GitHub issues. What state should they be in? It calls into question how they are closed or why they were closed. The other issue is that GitHub issues can be edited and comments can be deleted. I am not sure if it is healthy to politicize the state of a GitHub issue. Allowing the medium of rejection to affect the validity increases the chance of deflection, especially when we have multiple communication channels. A rejection should be a rejection and the reasoning can be examined afterwards. There was a problem hiding this comment. On Thu, Jun 09, 2016 at 05:07:44PM -0700, Stephen Day wrote:
I agree, but I don't think the current wording politicizes the state |
||
|
|
||
| **Timelines:** Specifications have a variety of different timelines in their lifecycle. In early stages the spec SHOULD release often to garner feedback. In later stages there will be bug fix releases, security fix releases, and major breaking change releases. Each of these should have a different level of notification. | ||
|
|
||
| - Pre-v1.0.0 specifications SHOULD release on a regular cadence and MUST follow the normal release process. In practice this should be a release every week or two. | ||
|
There was a problem hiding this comment. A (bi)weekly cadence goal has never been met by an OCI project (as far as I'm aware), and it would mean you'd have to open your week-long vote for 0.2.0 immediately after cutting 0.1.0. Can we make it “SHOULD release every month”? |
||
| - Major specification releases that introduce new base or optional layers, break backwards compatibility, or add non-optional features MUST release at least two release candidates spaced a minimum of one week apart. In practice this means a major release like a v1.0.0 or v2.0.0 release will take 3 weeks at minimum: one week for v1.0.0-rc1, one week for v1.0.0-rc2, and one week for v1.0.0. Maintainers SHOULD strive to make zero breaking changes during this cycle of release candidates and SHOULD add an additional release candidate when a breaking change is introduced. For example if a breaking change is introduced in -rc2 then a -rc3 SHOULD be made following the normal release process. | ||
|
There was a problem hiding this comment. We are trying to build a specification here, not a webapp. These timelines are way too tight. Larger companies are going to take time to allocate resources and examine the specification. Time needs to be given to take appropriate community feedback. There was a problem hiding this comment. @stevvooe This sets up a 3 week minimum guideline assuming zero changes. Community feedback requiring changes adds additional time automatically. And the RC releases will come off of what are already regularly scheduled pre-releases leading up to the RC. Can you be a bit more constructive? What do you think the minimum time from RC to release should be assuming the community has already seen many earlier releases? There was a problem hiding this comment. On Thu, Jun 09, 2016 at 05:26:41PM -0700, Brandon Philips wrote:
It didn't make it into the minutes, but I vaguely remember @RobDolinMS However, it is a long time to freeze development, so I suggest There was a problem hiding this comment. I agree with @stevvooe, these timelines are a bit tight even for software projects. A specification bump is useless without running code, so we have to accommodate the fact that we need some cook time with an implementation. One week for an This section is about specifications, but it reads like a tight timeline for OCI applications. Maybe make each rc a 2 week window with the final release being a 4 week window. There was a problem hiding this comment. It will be useful to identify a few key stakeholders beyond just maintainers to validate a release candidate. Just having one or two implementations doesn't seem to satisfy the requirements of a standard. Would it be possible to come up with a list of partners who have a stake in this effort and get their feedback before making a final release? If we do end up identifying some partners, then we will have to give them sufficient time to validate a release candidate. For large organizations, I would not be surprised if this process takes a month. There was a problem hiding this comment. On Fri, Jun 10, 2016 at 12:12:29PM -0700, Vish Kannan wrote:
Isn't there already a list of pretty icons for that 1? I think the
A month (4 weekly rcs + a final release) works for me 2, but in that There was a problem hiding this comment. @stevvooe @RobDolinMS Are y'all OK with a 1 month minimum time to make a major release? That would be a minimum of 1 week between releases (I am highly doubtful we will ever hit this best case) and an rc1, rc2, and rc3, and final. If someone shows up three weeks into the process, in rc3, with a show stopper everything would start over with rc4, rc5, rc6, and final. Making it 7 weeks. Ideally we would learn of show stoppers early in the months leading up to the RC. And the point of making regular releases is to build consensus early in the process instead of creating an explosion of activity at the end. There was a problem hiding this comment. @philips A 1 month minimum sounds okay. I would still like to hear from the larger companies on this. |
||
| - Minor releases that fix bugs, grammar, introduce optional features, tests, or tooling SHOULD be made on an as-needed basis and MUST follow the normal release process. | ||
|
There was a problem hiding this comment. The current image/runtime specs have two consumers: runtime maintainers and runtime users. If we're semantically versioning for the users (e.g. config authors and runtime callers), then “we've added an optional field to config.json” (to use a runtime-spec example) would trigger a minor bump. However, for runtime maintainers, that user-optional field is a new requirement that they MUST support in order to be compliant with the spec. Since folks who are likely to maintain runtimes are also OCI Members, I suspect we want to take the same care with minor bumps that we take with major bumps. There was a problem hiding this comment. Unless "optional features" means that they are "SHOULD"s rather than "MUST"s? Like extensions to the base spec? As far as I'm aware, we don't codify extensions in the spec at the moment (and I'd hope we wouldn't start -- let's not make this spec into IRC). There was a problem hiding this comment. On Thu, Jun 09, 2016 at 10:02:07PM -0700, Aleksa Sarai wrote:
The runtime-spec has seven SHOULDs in config.md and config-linux.md as |
||
| - Security fix releases MUST follow a special release process that substitutes the dev@opencontainers.org email for security@opencontainers.org. | ||
|
There was a problem hiding this comment. If the concern here is leaking security holes to a public dev@ list, I'd recommend giving this the bold you're using as a quasi h3. You don't want folks to miss it in the excitement of a security fix. There was a problem hiding this comment. I also don't understand whether or not we're meant to announce security fixes on There was a problem hiding this comment. I think the members of security@opencontainers.org team can figure out and coordinate the disclosure process to dev@. If someone wants to document this security release process we can handle it in another document as I don't think it is in the critical path to clarifying our overall release process here. |
||
|
|
||
| ## Conformance/Testing and Applications Releases | ||
|
There was a problem hiding this comment. This section is much shorter than the specification section. We should definitely codify rc releases -- this is quite important for several reasons. Currently we have runc Personally, I'd like it if we specified something like this:
There was a problem hiding this comment. On Thu, Jun 09, 2016 at 10:11:28PM -0700, Aleksa Sarai wrote:
For example…? The reason why it's shorter/simpler is that OCI Members There was a problem hiding this comment. I'm confused -- why is this in here at all then? runC is an OCI project, so we should be also making the rules clear for runC as well as the *-spec projects. If we don't want to make any rules for runC then we should remove the "Applications" from the title. No rules are better than half-baked rules. There was a problem hiding this comment. On Thu, Jun 09, 2016 at 10:30:22PM -0700, Aleksa Sarai wrote:
I'm all in favor of fully baking the rules. I'm just trying to |
||
|
|
||
| **Making a release:** OCI application projects MUST announce intentions to release with two project maintainer sponsors on the dev@opencontainers.org mailing list. After the announcement at least one more project maintainer MUST reply to the list with an LGTM within two business days for the release to be approved. The maintainers SHOULD wait two business days for maintainers to reply and review. If all maintainers reply with an LGTM then the release MAY release earlier. | ||
|
There was a problem hiding this comment. I hate to be silly: what is the definition of a "business day". When does it start? There was a problem hiding this comment. On Thu, Jun 09, 2016 at 05:13:59PM -0700, Stephen Day wrote:
I don't think we want to go into that ;). Can we just make this a There was a problem hiding this comment. @stevvooe Not to mention the fact that some of us are in bad timezones. :D There was a problem hiding this comment. It also should be a MUST, and it should definitely be a week. There was a problem hiding this comment. Are the @opencontainers/runc-maintainers OK with a week? |
||
|
|
||
| **Rejecting a release:** A project maintainer MAY choose to reply with REJECT and MUST include a list of concerns filed as GitHub issues. The project maintainers SHOULD try to resolve the concerns and wait for the rejecting maintainer to change their opinion to LGTM. However, a release MAY continue with a single REJECT. | ||
|
There was a problem hiding this comment. What happens when there are two REJECTS? Do we stay in the "resolve the concerns and wait" reconciliation loop? Is there a timeout where the issue is then raised to the TOB? There was a problem hiding this comment. On Fri, Jun 10, 2016 at 12:25:11PM -0700, Diogo Mónica wrote:
I think this should be “at their discretion, any maintainer may appeal There was a problem hiding this comment. I think the best course here so things are easy to understand is that we say 2/3 LGTM vote for a release is sufficient. But, rejects should be seriously considered and time and discussion made to change the REJECT to an LGTM. I will fix this up in a follow-up commit. |
||
|
|
||
| Security fix releases MUST follow a special release process that substitutes the dev@opencontainers.org email for security@opencontainers.org. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe a different criteria for a major vs minor release will help. For e.g. 0.1 should require LGTM from half but 1.0 should require 2/3 LGTMs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The last sentence reads a bit strangely around the MUST to me. How about replacing the last two sentences with:
And that also gives space for @mrunalp's different thresholds.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Jun 09, 2016 at 01:48:30PM -0700, Mrunal Patel wrote:
What about 1.1.0? The charter's only Member out for “I don't like
that release” is “then you can quit” 1. If that applies to minor
and point releases (and in the absence of wording to the contrary I
expect it does), then I think we want to be cautious about them as
well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A release should be preceded with a release proposal, giving reasoning about how the release meets the requirements of the release. If any requirements for the release are not met, but the proposal is to proceed despite the lack of met requirements, this should be called out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would also request that we get a LGTM from one of the platform specific parties for "Making a release". As in atleast one LGTM for Linux , one from Windows and one from Solaris or whoever is contributing to the specified project. We can keep a time limit for this response as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Fri, Jun 10, 2016 at 12:17:22PM -0700, Abhijeeth Nuthan wrote:
I don't think there's a need to call that out. Platforms who wish to
be represented should have maintainers they trust (and/or employ) on
the project, and expect the maintainers to respect any per-platform
concerns raised (just as concerns from anybody should be respected,
#17).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree we don't need to call this out. Maintainers should be trusted to make the right call, and platform maintainers should make their case to the relevant channels. It is pretty clear how this stuff should work based on the OCI gov. docs 5.d: "The primary means for any organization to influence the technical direction of the OCI Projects is via contribution or service as maintainers. "