Big thoughts for 4.0

[darrelmiller]

We would like to take an opportunity to discuss what might be some big, radical changes that could be done in a 4.0 version that would address issues with OAS that we have not been able to address with past changes.

Let us know what major changes you think would make a significant improvement to the way you describe your APIs.

Updated request: Since we converted the issue to a discussion, it allows us for better threaded and voting capabilities for different ideas. If you have several suggestions, consider separating to separate replies so those could be considered individually.

[swaldron58]

Bear with me on this one as it is not a change to the OAS per se, but a potential change in the way the spec or what may become specs (or types of extensions) are organized.
On board with the call for “simplification” and usability of the spec with the underlying caveat most have, “as long as the spec does what I need it to do, no matter how complex that is”. I don’t need to repeat that driver for change.

It’s groundhog day for me with this recurring lifecycle of the evolution of specs and languages from simple, to addressing more complex but needed capabilities, to “the spec is too complex and hard to understand, let’s start over”. I started in 360 assembler and 6 bit line protocols (P1024B to be specific) and everything since. Yes, some fundamental changes since then like moving away from SMP and procedural compute models to the current forms of shared nothing which has been huge value. But so many mistakes repeated again and again. I will also note that I view things more from an IT leader managing thousands of developers on hundreds of projects. The real world needs to run a successful and profitable IT shop. The need for velocity and predictability of outcome (quality, function, time to market, etc.). Specs are used by developers to solve a point problem. Standards are enforced by IT leaders when they lead to real business value.

One of those concerns is watching increasing use of schema as a programming language. Embedded business logic in what was intended to be CRUD messages to a discreet business object. This has always happened before hence nothing new, however it leads to a lot of instability, complexity and cost. Note that I am not suggesting we start taking things out of the spec to avoid improper or at least unadvisable use. My suggested change is in what we can do to help the community not shoot themselves in the foot quite so easily.

In programming languages we are all used to the migration to higher level languages. The benefit being providing a format where average developers can express the business functions needed and let a complier produce the runtime. There was always the ability to drop back into a lower level language for edge cases. There are tools out there doing this or at least moving towards the same for schema. I said schema instead of JSON or anything specific as hopefully the tools are not tightly bound to any one instance of a format.

My change is for OAI to overtly support this trend. Attack simplification by helping any tool provider coming up with a better way for developers to describe the 90% of the business needs at the API level and get developers away from manually updating schema. Compliers and similar approaches can do a lot better at leveraging the OAS to meet business needs with a lot fewer mistakes, faster to market, less fragile, hence lower costs. What can we do that makes it easier for a tool supplier providing a simpler interface to produce OAS compliant documentation? A means that negates the need for manually updating and managing JSON schema? This would still be API first, just not at a level so error prone.

As a refence point I recently ran a large development shop using SAFe coupled with model driven development. The business objects and services defined in the model. A complier produced .xsd and OAS 3.0 where there was little need or opportunity for anyone to touch the schema. One service definition in the model could begat numerous (100s even) of resource names (APIs deployed). Didn’t care about the number of APIs as wasn’t trying to manage individual schema or even crazier, snippets of schema. There was no need to make omnibus APIs to save on development work. Who needs multipath statements when I can just define another facet of the model for the unique need of each path? My point being left shifting the development work away from working at the schema level works. A simpler schema is not the answer. It’s how can average developers define business objects together with what actions are to be taken against them. Then let tooling produce the needed artifacts to deploy using a spec such as OAS 3.1.

[SensibleWood]

My change is for OAI to overtly support this trend. Attack simplification by helping any tool provider coming up with a better way for developers to describe the 90% of the business needs at the API level and get developers away from manually updating schema.

I 100% agree with this. The salient API-first approaches still concentrate on the primacy of the OpenAPI document as the vehicle for specification development, which is what many developers rally against from a code-first perspective. Being able to drive this from a common modelling approach, which is still API-first but removes the dependency on the OpenAPI document would be a big boost. The modelling approach would help drive the API specification, which then drives the code. That extra layer of abstraction would be really helpful for lots of folks, and allow for commonality in tooling on how to drive it.

[xiakunhou]

I would agree that auto-generation would save effort. While from my experience generating OpenAPI documents from code, there are many pitfalls,
When coding first, the developer prefers to reuse the model, while, usually they abuse "DRY" or over-designed.

For example, one response model is used by many endpoints, while some of them do not use some fields in this model; the Developer may abuse "inheritance", which results in unexpected field exposition. Code still can work, but all of this would generate unexpected documents.
Even though, this can be resolved by better design, “CQS”, "LSP". But in too many cases, bad codes are written.

Auto generation but not from code would be my choice.

• Code should depend on Schema, otherwise not allowed.
  

Besides, code could be generated from the schema, this should be a promising direction.

[rafalkrupinski]

And what would stop developers from writing bad OAI specs?

[xiakunhou]

"bad" code, especially, code with side effects on documentation. When handling OAI specs/code directly, less like to write incorrect specs/code.
Code and documentation are reviewed separately and they have a separation of concerns. If document generation from code directly, it would introduce complexity and back forth changes.
To simplify, it should be,
automation->schema(->code) instead of code->schema

[patricekrakow]

"Merge" with AsyncAPI in order to have one (and only one) specification language (and therefore, hopefully, ecosystem of tools) to define/document both synchronous (request-response) communication patterns and asynchronous (publish-subscribe) communication patterns for APIs.

[karelhusa]

Unifying sync and async API specs would bring great benefits:

• We would reuse most of the design skills in both areas.
• APIs could combine synchronous and asynchronous operations.
• There would be a single lifecycle (versioning) for both, therefore easier to follow for tooling.

AsyncAPI "wants to keep as much compatibility as possible with the OpenAPI Specification" anyway, see the AsyncAPI disclaimer".

[rafalkrupinski]

I find the tools out there already rather poorly supporting openapi as it is, I don't think it needs to be made twice as complex.
Instead I'd suggest making them sister projects, developing and versioning them together and in sync.

[patricekrakow]

Remove the servers property in order to have the OAS solely focusing on the definition of the interface, not giving any details about the implementation(s) of the interface. This is a bit "on the edge", but I think the servers property is crossing the boundary between the interface and the implementation.

[adamaltman]

Where do you draw the boundary? What about the security schemes? Those could be coupled to the servers in some way? What else may be part of the "environment" and not the interface?

It seems this could lead to standard unofficial specifications like x-servers because they are used by various tool providers.

[mrentsch65]

The servers property is especially useful for separating the addressing part of the endpoints url from the functional parts. I would not like to miss this powerful conceptual approach.

[rafalkrupinski]

I think OPs idea to treat the addressing part as out-of-scope and remove it completely.

IMO the idea has some substance. The server list has a limited usefulness. In my experience it typically has a production and testing servers, which needs to be selected by the developer anyway, depending on run environment.
The only two useful things I can do with it is to use the first entry as the default and issue a warning when the user attempts to connect to a server not on the list.

[handrews]

This is being discussed in terms of a separate Deployments / API Shapes objects at:

• Deployments independent of API descriptions #66

[reloxx13]

I18n
OAI/OpenAPI-Specification#274
Also reopen the above issue as its not duplicated as explained multiple times now.
Content I18n != Documentation I18n

[nacho4d]

I cannot stress this more. Products that have global teams, global audiences, global users will be benefitted.

Please See this issue OAI/OpenAPI-Specification#274 (comment) ← Specially this comment) for details about "Content I18n != Documentation I18n"

[jeremyfiel]

I'm not sure adding a headers map to the Operation object is the right move, but I really want some clarification on why reusable headers are not allowed across requests and responses. This is a very confusing area of the spec. If I have a header used in Request and Response, I expect to be able to $ref my components>headers entry. Why should I need to duplicate a parameter object for my request header?

We have many GET/DELETE operations requiring custom headers in the Request and it's not currently possible to reuse my components/headers in these operations because a headers map doesn't exist on the operation object and we don't allow request bodies on these operations.

[handrews]

• Rework the whole content encoding area. I integrated the JSON Schema content* keywords for 3.1, but it was a bit of a rush job and needs cleanup. I hope to contribute basic cleanup for 3.1.1, and maybe some slight improvements would fit in a 3.2, but really there's now a lot of conceptual overlap and weirdness about how that is handled and it would benefit from a comprehensive re-think for 4.0.

• Add annotations for additional client-server semantics to OAI's JSON Schema extension vocabulary, or perhaps as their own vocabulary. This would be better done through OAI than JSON Schema IMHO. It's a somewhat trickier area than it initially looks primarily due to people wanting things to be required or not depending on message direction, which doesn't fit nicely into the JSON Schema validation model. But there are some ways to work with that to clarify things like "you can't change this and we'll ignore it if you try" vs "you can include it in a PUT if it's unchanged from the state marked by your ETag, but if you change it it's an error" vs "if you include this in a PUT or PATCH at all it's an error", etc. Also create-only fields, write-once fields (create-only is a special case where the write-once has to happen on create), and probably other stuff I'm forgetting.

• Concise representation-oriented syntax. As a system for describing any HTTP API, OpenAPI's flexibility is a strength, but it comes at the cost of a lot of boilerplate if your API uses regular, predictable patterns for interacting with a representation. I should just be able to say "this is my representation schema (using the annotations from the point above), here's the URI template for the item and collection, and GET/PUT/POST/DELETE work as expected, and PATCH works with patch media types X & Y (which allow the PATCH request schema to be derived predictably from the representation schema). There are varying levels to which this can be done, e.g. automatically deriving PATCH schemas is more of a bonus than essential. This does require being able to specify what your "normal" item and collection interactions look like, which is more of a concern for collections.

• Consider reworking parameter description into a single schema rather than per-parameter descriptions, so that interactions among parameters (and possibly also headers? idk) can be described with JSON Schema. This sort of interaction-describing functionality is requested regularly IIRC, and JSON Schema already has all of the necessary features.

[handrews]

@adamaltman yeah, when I made the comment it was on a GitHub issue, so I wasn't thinking about threads. I can make four new comments and strike this one maybe? I've barely ever used discussions so I have no idea what the capabilities are.

[webron]

That would be great, @handrews!

[handrews]

@webron I do intend to come back and split this up and elaborate on them, I'm just a bit distracted by other things for the next week.

[ralfhandl]

@handrews looking forward to chime in on the „item and collection“ pattern (bullet point 3), combined with the „create/update vocabulary“ (bullet point 2). We literally have hundreds of APIs where the collection-item pattern is explicitly spelled out as two paths with five operations, only differing in the pair of collection/item path templates and the triplet of create/update/read structures, which are projections of an „entity“ super-structure.

[handrews]

@ralfhandl I'll try to get to that soon. Life has not been cooperating recently, unfortunately, and I have had limited bandwidth which has been consumed by relatively urgent goings-on with JSON Schema.

[karenetheridge]

I've raised this on slack, so I guess I should raise it here as well:

• rethink/rework the style/explode behaviour. It's a lot more clear how to use these values when serializing a data structure into a request, and much less clear when dealing with a http request/response and trying to deserialize into a data structure that can have a json schema applied to it.
  • the Swagger docs (https://swagger.io/docs/specification/serialization/) imply that the deserializer needs to peek inside the json schema at the 'type' keyword to see if it is desired to deserialize to an array or an object or a number (or an array of numbers or an object of numbers) which gets hairy really quickly and not really doable after the first level of an object/array if the json schema has any sort of complication to it at all (e.g. nested schemas inside allOf, etc).
  • Some headers can be used twice and probably should be parsed into an array for validation (but the spec doesn't talk about that). Some headers aren't parsed using the default style=simple, explode=false mechanism (e.g. Set-Cookie only uses separate header values and individual entries cannot be split; Cookie is split by ;\s*, multiple values are not allowed for Content-Length).
  • Additionally I believe it is already possible to parse all query parameters into one "parameter value" as an object (which is something people have been asking for) using a particular combination of style/explode/schema values, so an example of this should be provided in the spec.
  • use of style/explode shouldn't be allowed in request or response bodies at all - we already have media types for this.

[bali182]

Completely agree, the parameter serialization is one of the most frustrating parts of OA to deal with, especially the multiple variant of complex type parameter serialisation (why does this exists, I'd seriously like to know the historic reasons).

I have a kind of wholistic serialisation/deserialisation implementation in typescript/javascript. I have a good amount of tests for it, but I'm sure it's not 100% perfect. You can find it here if it helps:
Source: https://github.com/oats-ts/oats-ts/tree/master/projects/openapi-parameter-serialization
Npm: https://www.npmjs.com/package/@oats-ts/openapi-parameter-serialization

It's not dealing with JSON, but it tries to deal with all other standard parameters, as far as I understand the docs. For cookies only primitives are implemented, exactly by the serialization reasons you describe.

[handrews]

use of style/explode shouldn't be allowed in request or response bodies at all - we already have media types for this.

Which reminds me, we can apply media types to the entire query string. The one that everyone thinks is "standard" (but is not actually part of RFC 3986 at all) is application/x-www-form-urlencoded, which easily maps into the JSON data model and therefore could be described by a single schema rather than partitioned out by parameter.

Furthermore, doing so allows use of the content* keywords to do things like embed JSON values (although i guess that can be done on a per-parameter basis as well if you want to).

To address some of @karenetheridge 's concerns about explode, and also thinking about issues like #1706 ("deep objects"), it seems to me that a better solution than increasingly complex control keywords is to define additional query-string-friendly media types that map into the JSON Schema data model (or potentially other schema types- really, mapping to a data model is the important part, then you can describe model however you want).

This could either be a hybrid thing like using JSON Schema's content* to embed other media types inside application/x-www-form-urlencoded or you could do something more direct with the entire query string. Hmm.. can the necessary URL encoding be considered a content encoding of some sort? OK now I know I'm going to mix up content codings, content ENcodings, transfer codings, etc. so I'll stop rambling. My point being that there are a lot of ways to support complex query parameters and taking a step back and re-thinking it more thoroughly, as I think @karenetheridge is proposing, seems like a good idea to me.

[darrelmiller]

Consider flattening the responses object to an array that uses pattern matching to define the corresponding response object.

  /pet:
    get:
       responses:
         - criteria:
             status: '200'
             mediaType: application/json 
             conditions:
               "$/response/body#/petType": dog
               "$/request.path.format": json            
           description: Success
           schema:
            type: object
            properties:
              name: 
                type: string
              barkVolume:
                type: number

This would significantly reduce the nesting for simple API designs, provide support for more sophisticated interactions. It would allow mapping request values to specific responses, which is a common feature request.

[adamaltman]

Can you help me understand the keys in this object? (I don't know that I understand the $, /, and . symbol meanings.)

               "$/response/body#/petType": dog
               "$/request.path.format": json            

[RomanHotsiy]

I wander if those conditions should be combined using logical AND or OR?

[darrelmiller]

@adamaltman I borrowed the syntax from the way links work. The idea is that you can reach in the request or response object to perform tests.
@RomanHotsiy Yes, those conditions would be a logical AND. A response would need to match all the the conditions.

[handrews]

Objects vs arrays consolidated here:

• Consideration for moving from objects to arrays. #32

I have not verified whether all ideas here appear there, feel free to add to that separate discussion.

[adamaltman]

Consider Operation Examples

In 3.1, we can describe examples for media type objects, individual parameters, schemas, and schema properties.

However, we can't describe a cohesive operation example (for example, this request parameter with that request body leads to these response headers and this response body). This makes the current mechanism of describing examples sometimes incongruent.

[adamaltman]

Also, I'm thinking of a big 4.0 thought. What about redoing the operation object completely? I'll post an idea about that another time.

[jeremyfiel]

We are currently brainstorming something similar to this where we are referring to them as scenarios. The idea that a particular request can produce a particular response based on business rules or certain parameters. I would be interested in seeing this evolve a bit more.

We typically have a relative folder with many different examples but there's no way to link them

[mikekistler]

I think that you can use named examples to link together example parameter values, request body, and response body -- forming a cohesive operation example.

For example, you can define an example with the name "Create a foo" (in examples) for appropriate parameters, the request body and the response body. Taken together, these can be considered a cohesive example for "Create a foo".

[adamaltman]

I like that @mikekistler ... it would be by convention, and not by specification?

[mikekistler]

I think right now it is by convention, but few seem to know about the convention. So I guess I'm proposing that recommend this explicitly in the spec, though maybe not require it.

Taking another angle -- it would be great to have a way to describe "whole examples", consisting of parameters, request body, and response body. One way to do that is with the convention I described, but I'm not wed to that. I'd gladly accept some other solution if that's what folks prefer.

[adamaltman]

Allow duplicate paths

(Note: I'm sure there are issues on this already, but I couldn't find them. Sorry.)

Situation

OAS <= 3.1 does not allow for duplicate paths. It makes some of these use cases more difficult:

• RPC-type APIs (like JSON RPC) where there are many "methods" but only one path. (These are different APIs in concept.)
• Evolving APIs where it is possible to describe in OAS <= 3.1 through polymorphism. This adds some complexity at the schema level, and often at the cost of clarity -- for example, it is not like you can describe Request A will result in Response A, and Request B will result in Response B. Instead, it's anyOf or oneOf or it requires an additional discriminator property. (These are different versions of the same API in concept.)

Complications

• Paths are the map keys, and therefore can't allow duplicate paths (as that would be invalid JSON).
• For any given PathItem, the http method (e.g. get) is the path item's operation map key. There can't be two operations defined for any given http method.

Ideas

1. Make a new pathId key with the path in the path mapping key. The path can be a property of the path item.

   paths:
     anExamplePath:
       path: /samples
       get: 
          ... 
     anotherPath:
       path: /samples
       get:
          ...

2. Make paths an array. add a path property to the PathItem object.

   paths:
     - path: ./samples
       get:
           ...
     - path: ./samples
       get:
          ....

3. Redesign PathItem to allow multiple operations with the same http method. This means probably moving the http methods into the operation schema by adding a new property operations as an array of operations.

   paths:
     /samples:
       operations:
        - method: get
           ...
        - method: get
           ...

My personal preference is the last option to keep the paths more organized.

I do think there is opportunity for further flattening. For example, move all props current in the path item object down into the operation object. Flatter, with more possible repetition, seems better than deeper as far as what seems like newer people understanding OAS. In other words, simplify by eliminating the path item object and making it an array of operations.

[rafalkrupinski]

In dynamic languages, endpoint evaluation will take longer if oneOf matching is done based upon endpoint input data for the same path.

How does the delay compare to time spent in IO? In the extreme cases where it does matter, you just don't use it.

How common is this endpoint overloading and should it be a general case when schema oneOf can result in working code today?

Very hard to say just from analysing the specs in the wild, since it's impossible to express it. But function overloading is rather popular in programming and one thing stopping devs from using it is the lack of explicit support in OAS.

If we did this my preference would be to have the http verb be the key and then have a list of implementations.

Post an example, please.

[spacether]

If we did this my preference would be to have the http verb be the key and then have a list of (oneOf) implementations. That keeps the verbs as unique keys and avoids the parameters array + in keyword issues that we currently have.

An example

paths:
  /samples:
    operations:
      get:
       - ...
       - ...

This makes it easier for tooling to loop over implementation in get and group http verb implementation functionality together.

[mabar]

Why does OAS support oneOf? - that's already overloading, only on data level, instead of path level.

oneOf is useful for complex data structures.

operation either accepts and returns a single object or a list

This is usually solved by generics. Practically the proposed duplicate paths feature. That is a different feature than XOR operator (oneOf)

[rafalkrupinski]

@mabar , looks like you forgot to add empty lines between quotes and answers, but, after giving the whole idea a second thought, I think I agree with you.

[handrews]

The map vs array topic has been consolidated in its own discussion, while generics are being addressed in 3.x:

• Consideration for moving from objects to arrays. #32
• Recommend describing templates/generic types using $dynamicRef OpenAPI-Specification#3601

I haven't verified if every map vs array idea here has been addressed in the other discussion, so feel free to copy-paste relevant bits. I don't think there's a separte discussion for the rest of this thread.

[adamaltman]

Move securitySchemes out of components

Everything in the components object is used with the reference object ($ref)... except securitySchemes which are used by name in the security object/list.

This inconsistency makes it both harder to explain components and use security schemes to new users of OAS.

I could be missing something such as security schemes being reused by referencing other OpenAPI documents. However, there is already another solution for that (the reference object can reference external files too and not only the components object).

openapi: 4.0.0
securitySchemes: ...

[karelhusa]

I prefer keeping the securitySchemes inside components and allowing to apply them by $ref as for other component types.
The security area (securitySchemes and security) seems inconsistent with other OAS elements, where I can either define the component (parameter, schema, etc.) in place or use a reference.
With such an approach, I could share security schemes between multiple APIs (microservices) by referencing the same OAS file.

[flaksp]

Represent servers as a map instead of array

Currently, servers is kinda tricky to use for code generation purposes because each server entry does not have any short identifier. There are only two identifiers: its index in array and url.

In case of using index as identifier we may generate following pseudocode:

const SERVER1 = 'https://dev.api.example.com/';
const SERVER2 = 'https://stage.api.example.com/';
const SERVER3 = 'https://api.example.com/'; // it's production

If we use url as identifier, generated code may look like this (but it will look ugly for long URLs or URLs with Server Variables within):

const SERVER_DEV_API_EXAMPLE_COM = 'https://dev.api.example.com/';
const SERVER_STAGE_API_EXAMPLE_COM = 'https://stage.api.example.com/';
const SERVER_API_EXAMPLE_COM = 'https://api.example.com/'; // it's production

Probably it could be better to represent servers as a map instead of array:

servers:
  development: 
    url: https://development.gigantic-server.com/v1
    description: Development server
  staging:
    url: https://staging.gigantic-server.com/v1
    description: Staging server
  production:
    url: https://api.gigantic-server.com/v1
    description: Production server

It was already mentioned by the community: #1821, #2229 (duplicate of 1821).

(tbh I like the idea to remove servers keyword more than this)

[earth2marsh]

The examples given with dev/stage/production... note that there are times where the contract itself may change between those lifecycle stages. In that case, I'd say different documents are more appropriate, or dropping servers entirely. In case it is not defined, assume the server is where the document was found.

[ralfhandl]

the server is where the document was found

Make it a strong recommendation that the document can be found at api-root/openapi.json or some other well-known place.

[handrews]

Maps vs array discussions have been consolidated at:

• Consideration for moving from objects to arrays. #32

I have not verified whether everything in this thread is covered there, feel free to copy things over!

[RomanHotsiy]

Maybe an unpopular opinion but I would vote against any "big, radical changes".

It looks like tooling authors can't keep up the pace of the changes. I see too many people still stuck on 2.0 because "tool X doesn't support 3.0".

I believe this is a result of series of huge changes: 2.0 -> 3.0 and, 3.0 -> 3.1 (the latter one doesn't seem like a big change but the new JSON Schema version has much more complexity for tooling authors beyond data validation).

I'm not against breaking changes. They are needed for sure. But I would focus on small quality-of-life-improvement changes that won't introduce too much complexity for tooling authors (like this or this).

[handrews]

We're planning those sorts of changes in parallel 3.x releases. Please add suggestions for minor or patch releases here:

• 3.x.y patch release approach OpenAPI-Specification#3528
• 3.x minor release approach OpenAPI-Specification#3529

[bali182]

Allow describing the set-cookie header

I edited this proposal based on the conversiation with @flaksp, so the idea comes through cleaner.

Currently with the cookie typed Parameter Objects we can only define expectations for the Cookie request headers in HTTP requests. This is useful for 2 things:

• In case you are calling APIs from a non-browser environment, this tells you that a given operation expects cookies.
• On the server-side code generators can implement cookie deserialization based on current spec.

The problem

• In case an operation expects cookies, there is no way to figure out who sets said cookie.
• There is no way to assist in generated server code, what operations should set what cookies, and can't take care of spec compilant serialization automatically.

Proposed solution

Based on how the headers are defined in Response Objects, I'm proposing a new cookies field for the Response Object, of Cookie Object type. The Cookie Object follows the same philosophy as how Header Objects are defined, except in is always cookie, and style matches cookie serialization style(s).

Example:

{
  "paths": {
    "/login": {
      "post": {
        "operationId": "login",
        "responses": {
          "200": {
            // Here the server produces the cookie in a response
            "cookies": {
              "someToken": {
                "required": false,
                "schema": { "type": "string" }
              }
            }
          },
          // No cookies are being set here
          "401": {}
        }
      }
    }
  },
  "/protected": {
    // Here the server expects the cookie in a request
    "get": {
      "parameters": [
        // Matches the response cookie by name, 
        // can be identified that the login operation will produce this cookie.
        {
          "in": "cookie",
          "name": "someToken",
          "required": true
        }
      ]
    }
  }
}

[bali182]

In that case, you may describe both cookie parameters and Set-Cookie response header for an operation.

The issue with this is, that the actual "semantics" of the Set-Cookie header (I'm talking about this) cannot be described properly with what's available in responses -> <status> -> headers. If I say Set-Cookie is a string typed response header in my OpenAPI document, that ignores the whole "sub-protocol" that comes with using cookies. I may want to set multiple cookies with different names, but I have no way of describing this currently in OA.

But I may be approaching this the wrong way. Could you describe what's the point of currently existing cookies? The main user of cookies on the request side (browser) cannot really use it as the Cookie header cannot be set from the browser manually.

So I assume cookies currently are meant for:

• Client side: If you are making HTTP requests from a non-browser environment, know that the server expects that the described cookies may be there
• Server side: it's an aid for parsing the Cookie header

Is this a correct assumption?

[flaksp]

The Set-Cookie header is is not designed for being read by an application — that's the reason why documenting its content is mostly pointless. For example, in a browser you can't access Set-Cookie response header with JavaScript for security purposes.

The typical and correct use case of the header is:

1. Receive it,
2. Remember its content into some store, let's call it the cookie store (browsers store it in document.cookie for you),
3. And send it back with each request in Cookie request header.

That's how HTTP abstraction layer works. And on application layer, you should read cookies from the cookie store (document.cookie in JavaScript) — you should not read them directly from HTTP response headers (even if your application is not a browser).

The main user of cookies on the request side (browser) cannot really use it as the Cookie header cannot be set from the browser manually.

It's possible to set cookies manually. It's interesting that you can't also send your custom Cookie header in JavaScript (also for security purposes). But you always can modify the cookie store. And the Cookie header sent with each request will always contain contents of the cookie store.

Is this a correct assumption?

Yes!

[bali182]

It's possible to set cookies manually.

I meant using traditional means. This is mutating a browser global, not setting the Cookie header. If you do fetch('<url>', { headers: { Cookie: '<cookie>' } }) it has no effect. But this is not really my point.

The Set-Cookie header is is not designed for being read by an application — that's the reason why documenting its content is mostly pointless. For example, in a browser you can't access Set-Cookie response header with JavaScript for security purposes.

I don't want to read it manually.

My main goal is:

• So that server side codegen at least able to assist with what cookies do I need to set for what operation, and take care of serializing them based on the spec.
• Client side code to know who's setting the cookies they need to provide.

It might be intentional, but with current spec we are saying

• We can set expectations for cookies
• But we are not telling you who's going to provide them

This is not only bad for what I mentioned (server side codegen), but for the users of your API too. If someone's using your API, sees that some operations expects cookies, the logical thing is to look for who's providing them. This information can't be made clearly available with current spec. This is the reason I'm proposing a change.

[rafalkrupinski]

Not all clients are browsers, and even then, cookies declaration should be there, if only for documentation purposes.

[bali182]

@rafalkrupinski Yes that would be my goal with this idea. I'm going to update the original wording so it's cleaner.

@flaksp Thank you very much for cleaning up how cookies work currently. Thanks to this conversation, I realized I have cookies implemented incorrectly.

[spacether]

How about limiting tag names to the same set of characters that component names are limited to (All the fixed fields declared above are objects that MUST use keys that match the regular expression: ^[a-zA-Z0-9\.\-_]+$). In code generation contexts, this will allow easier file (module) name generation and reduce the likelihood of filename collisions.

Also have tags be a map where the key is the tagName. That structurally prevents tag collisions.

[ralfhandl]

Lost a discussion with a product manager who insisted that tags are human-readable text, so now our tags may contain spaces:

[philsturgeon]

A lot of companies use tags for Docs generation (most docs tools in fact) so I can understand this. I recommend this too. Code gen people are sad, but then, that probably suggests the concept has been overloaded.

[lornajane]

I've got a formal proposal for upgrading tags with exactly this sort of thing in 4.0. I've suggested: summaries or other readable labels, nested structure, and a type (so we could use for nav, for lifecycle, and for things I haven't thought of). It would be good to get more eyes/ideas on it. #67

[handrews]

I've consolidated this sub-thread under:

• Consideration for moving from objects to arrays. #32 (comment)

[deefdragon]

This may be more appropriate for the json schema side of things (Im never sure where the line is), but one of the biggest issues I have consistently encountered is lack of generics support. Specifically, for things like paginated endpoints where I have a common data structure to return.

{
    "Cursor": {
        "Previous": 0,
        "This": 10,
        "Next": 20
    },
    "ItemCount": 10,
    "Data": [
        //one of N different objects.
    ]
}

To properly define this within my sachems, I have to re-define the pagination object every time, with the data set to the whatever I am paging in that endpoints case.

Another example that could use it would be structured error returning. I can return a general error with a message and code most of the time, but there are a few edge cases where I want to return some form of structured data in the error itself, and for each of those I have to redefine the wrapping error object.

[flaksp]

Generics are part of the JSON Schema specification already. It's called dynamic references. Check out "Using Dynamic References to Support Generic Types" article to get started.

[deefdragon]

Just to confirm, it looks like generics are part of schema 2020-12, which is part of openapi 3.1?

[flaksp]

Yeah, OpenAPI 3.1 uses JSON Schema draft 2020-12 to describe schemas.

[rafalkrupinski]

What's wrong with putting Cursor schema under components/schemas?

[spacether]

One can do this earlier than 3.1 by extracting the common fields into a PaginationBase and then doing

allOf: #/components/schemas/PaginationBase
data: your data schema or ref here

The generics do something similar where data still needs to be defined.

[jeremyfiel]

You can also check out RFC7807 for a problem+json schema error response.

…

On Wed, Feb 22, 2023, 01:17 deef ***@***.***> wrote: This may be more appropriate for the json schema side of things (Im never sure where the line is), but one of the biggest issues I have consistently encountered is lack of generics support. Specifically, for things like paginated endpoints where I have a common data structure to return. { "Cursor": { "Previous": 0, "This": 10, "Next": 20 }, "ItemCount": 10, "Data": [ //one of N different objects. ] } To properly define this within my sachems, I have to re-define the pagination object every time, with the data set to the whatever I am paging in that endpoints case. Another example that could use it would be structured error returning. I can return a general error with a message and code most of the time, but there are a few edge cases where I want to return some form of structured data in the error itself, and for each of those I have to redefine the wrapping error object. — Reply to this email directly, view it on GitHub <https://github.com/OAI/OpenAPI-Specification/discussions/2930#discussioncomment-5073313>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHU7MTMC3L2OPNJKRA5KRHTWYWVRLANCNFSM5WSVJSEQ> . You are receiving this because you commented.Message ID: ***@***.*** com>

[spacether]

Replace the Security Requirement Object with a Security Scheme Object with the following clarifications:

• Security Scheme Object can contain $ref to defining Security Scheme Object
• $ref must be set for the security requirement use case
• flows may be limited to only a subset of flows in the $refed schema if it is defined, otherwise all original flows are used

3.X.X security requirement

{
  "petstore_auth": [
    "write:pets",
    "read:pets"
  ]
}

Becomes:

{
  $ref: components/securitySchemes/petstore_auth,
  description: "only these two scopes are allowed because blah"
  flows: {
    "implicit": ["write:pets", "read:pets"]
  }
}

This would eliminate the unnecessary Security Requirement Object and have ref follow conventional patterns for Security Scheme

[spacether]

How about replacing server variable object with a schema object of type string? The Schema already contains enum, default, and description as fields.
The new definition of Server Object.variables would be Map[string, Schema Object]

[MikeRalphson]

Or just a single schema with properties which map over the server template url parameters, as is being proposed for query, path and header parameters. (And templated security Definition urls.)

[jeremyfiel]

I would like a (better) way to document a request and response body for a human readable metadata payload along with binary content.

Our use case is similar to ftp file upload or EDI centric behavior such as a server to server request where binary content is uploaded and a metadata JSON payload is also included alongside to describe the file.

From my understanding, the request body can only be one type of mediatype per request. In this example, We need to document both parts of the request, but there's no way to tie them together in the same way the Encoding object does for multipart/form-data.

{
    "requestBodies": {
        "sampleMixedRequestBody": {
            "description": "A description of the request",
            "content": {
                "application/octet-stream": {
                    "schema": {
                        "type": "string",
                        "format": "binary"
                    }
                },
                "application/json": {
                    "schema": {
                        "type": "object",
                        "properties": {
                            "fileName": {
                                "type": "string"
                            },
                            "fileSize":{
                                "type": "number"
                            },
                            "fileSaveLocation": {
                                "type": "uri-reference"
                            }
                        }
                    }
                }
            },
            "required": true
        }
    }
}

[jeremyfiel]

Additionally, the ability to document multiple, nested mediatypes within a multipart request or response as defined in RFC2049-Page15

Currently, there's no way to define a complex request or response for multipart/[mixed, parallel, alternative] in OAS.

A required parameter of boundary: <string> should be added to the mediaType schema when multipart are defined. min: 1, max: 70 per https://www.rfc-editor.org/rfc/rfc2046#page-22

The only mandatory global parameter for the "multipart" media type is
the boundary parameter, which consists of 1 to 70 characters from a
set of characters known to be very robust through mail gateways, and
NOT ending with white space. (If a boundary delimiter line appears to
end with white space, the white space must be presumed to have been
added by a gateway, and must be deleted.) It is formally specified
by the following BNF:

 boundary := 0*69<bchars> bcharsnospace

 bchars := bcharsnospace / " "

 bcharsnospace := DIGIT / ALPHA / "'" / "(" / ")" /
                  "+" / "_" / "," / "-" / "." /
                  "/" / ":" / "=" / "?"

[mikekistler]

Improvements for links in v4

I've put together a proposal for reworking/extending support for links in OASv4/Moonwalk.

https://gist.github.com/mikekistler/1983a8249c0df56a46f94b4df1b7181d

I'd be happy for feedback on this.

[handrews]

Separate discussion at:

• Improvements for links in v4 #58

[myronww]

I have added a proposal for an archetype field for paths. The archetype field would allow for the assignment or declaration of a design or architectural pattern to paths. This would be extremely useful for documentation, tools, and testing.

Motivation

When working with larger product API sets that span multiple features and teams, it is important to ensure that the "Paths" associated with the different features produce consistent patterns of behavior across the entire product API. It is also equally important that API Paths communicate any patterns they are attempting to implement to the tools that will work with the API reflection document.

When API path sets are tested, the tests need to ensure that the path, properly implements all aspects of specific archetypical patterns. For example, an api path that implements a 'editable-collection' pattern for users on '/v1/users/' might need to adhere to the following set of design rules:

• Provide a 'post' method for creating new collection items
• Provide a 'delete' method for removing items from the collection
• Provide a 'get' method for listing the items in the collection
• Provide a 'patch' method for updating an item
• Provide a 'put' method for replacing an item
• Return data in a standard collection form { items: [...], "total-count"=100, "page-count"=10, "page-num"=1, pages=10 }

Having the information about the archetype that an API is supposed to implement is critical for testing to ensure that the APIs are meeting architectural standards across the entire API.

Also, when API path information is consumed by a code generation tool that is attempting to create an interop client, the tool can utilize the archetype information declared for a path to help tune the generated code to better align with the intended behavioral patterns and functionality.

By providing archetype desciptions for paths, the OpenAPI document can play a vital role in ensuring that APIs are testable across larger API sets and that code generators have the informaiton they need to properly implement API interop patterns for APIs.

Proposed solution

The proposed solution is to add an 'archetype' field that is declared for a path. The archetype field provides a unique identifier that describes any special architectural design rules that a path should follow in its implementation and that provides detailed design information for code generation tool creators.

Examples

`
"/v1/users": {
"summary": "Path for working with the collection of users."
"archetype": "http://someorg.com/api-docs/design/collection.md"
}

"/v1/users/{user-id}": {
"summary": "Path for working with a single user."
"archetype": "http://someorg.com/api-docs/design/collection-item.md"
}
`

OAI/OpenAPI-Specification#3369

[handrews]

@myronww I think part of the issue here is that this "archetype" concept is not itself broadly used / accepted in a consistent way. It is difficult to add a concept to a standard when that concept is not, itself, a standardized concept.

The OAS describes many things that are based on other standards, most obviously HTTP. And then it allows many things that have no need for standardization, like a human-readable description.

While I at least more-or-less get what you mean by "archetype" here, I could not walk into any random API design team, ask them about the term, and expect them to understand it, much less use it "correctly."

Standards can provide interoperability in the details of a shared concept. But when a concept itself is not sufficienlty broadly used, a standard can't force it into being. Attempts to do such things tend to produce features that are poorly understood and either not often used, or often used incorrectly. This is why creating extensions and working to get adoption and consensus is a better way to introduce totally new concepts that do not come from other standards already.

[myronww]

@handrews I get what your saying. I would offer that one reason to add something to a standard is as a means of instruction. I wouldn't know half as much knowledge about REST API design if there were not standards around it. Also, if i went from one company to another, i would have to learn a new schema for every API.

I could as suggested go and put in a 'x-archetype' field in my own Open API document, but then I would be one of a few that had APIs that were super easy to inspect, test and code generated with.

My own API tests would easily be able to inspect APIs to make sure all teams were following consistent architectural standards. My code generation code would be simpler because of pattern hinting. But the point to a standard is to identify patterns and share solutions for those patterns and maybe offer some knowledge about how to improve the quality of APIs for others.

[mikekistler]

This may be tangential, but I'll just note that we've tackled the "archetype" problem in a different way with TypeSpec. We codify our archetype operations, models, parameters, etc in TypeSpec libraries, and API authors then use these components and get a conforming API "by construction". Anyone can create their own TypeSpec library to define their own archetypes, and it is pretty easy. You could define:

• createOrUpdate -- a PATCH on the resource URL that accepts a resource schema (JSON merge patch) and returns the same schema, with standard error responses, conditional request headers, telemetry headers, etc.
• read -- a GET on the resource URL, returns the resource schema, standard error response, etc.
• list and delete -- similar

You can see an example in our Playground

[myronww]

So what I think your saying is that you use TypeSpec to codify your specifications and then you can generate OpenAPI specifications from the type spec. This seems like a good approach if you have the resources to create a spec language that can generate specification document. You could even use inheritance to propagate proper patterns to all the URL paths.

The question i would have is, if there was an archetype field for OpenAPI, what would TypeSpec do with it? What would you write there when generating an OpenAPI spec from a TypeSpec API.

Would it be useful to communicate URL path pattern information from TypeSpec to other tools?

[handrews]

Separate discussion for this thread:

• Add 'archetype' field for paths #79

I have not verified whether everything in this thread has been migrated.

[jeremyfiel]

If you just want to link to an external markdown file, you can use `externalDocs` at the operation level and put any link you want.

…

On Sat, Sep 23, 2023, 20:34 Myron Walker ***@***.***> wrote: Remember, the type is offered by the 3rd party. The types are not part of the standard. What is part of the standard is a field for providing a type hint. That is all, the standard will not dictate what types to use and such. The implementer would use the field to define their own types. { "paths": { "v1/users": { "archetype": "http://someorg.com/api/types/pageable-collection.md" }, "v1/users/{id/": { "archetype": "http://someorg.com/api/types/collection-item.md" } } Its literally just a field so you can put in a type hint. Just like the methods have a 'operationId' hint that code generators can use. — Reply to this email directly, view it on GitHub <https://github.com/OAI/OpenAPI-Specification/discussions/2930#discussioncomment-7091373>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHU7MTJGPL3YT47JNNERA4DX3552PANCNFSM5WSVJSEQ> . You are receiving this because you commented.Message ID: ***@***.*** com>

[myronww]

The purpose is not just to link a doc, see above. The URL is just serving as a unique identifier in a different context. You could just as easily use a name like 'EditableCollection' as the identifier. Its a simple unique identifier to pattern assignment to offer a hint to tools. The identifier is arbitrary.

[razb-viola]

Add support for core TypeScript features, like:
Partial;
Generics;
Pick;
Omit;
Specific;
Enum item, like Statuses.Approved, instead of the entire Statuses enum;
Reference for specific properties within a schema, like Foo.bar.baz.

[spacether]

There are many consumers of Json Schemas. TypeScript code generation is only one use case of many. Openapi uses json schema to define schemas. I doubt that the JSON Schema committee would be open to adding keywords with typscript-only definitions/use cases.

[flaksp]

I was talking a bit with JSON Schema maintainers long ago about difficulties related to generating a code using the specification, and they said there is a project focused on this problem but it lacked human resources at that moment. I believe you can hop into their Slack and ask for some details about this, and probably offer some help, but it won't be as easy as just adding some TypeScript-related keywords into the spec.

And this is not OpenAPI's responsibility, it's all purely about JSON Schema, btw.

[razb-viola]

Hi guys, maybe I could explain it better: I am not looking for a tool to translate JSON schema to TypeScript.
I am looking for more rich keywords to be added to the OpenAPI Specification, so I can write better and reusable components.

I am not familiar so much with JSON schema, I just understood that the difference between this to OpenAPI Specification (oas) is that oas is a definition for the whole API, while JSON schema is just a syntax and data validation against a JSON input.

So to make my question or need clearer, this is the problem:

User:
type: object
properties:
id:
type: string
name:
type: string
minimum: 2
maximum: 20
password:
type: string

now I want to $ref specific part of the User schema:

account:
type: object
properties:
user:
$ref: Partial<User, 'id' | 'name'>

Notice that I referenced only the id and name but never the user's password.
In this example I used TypeScript, but without any mean to actually use TypeScript specific keywords or something, but the same idea.

When I write an API definition using oas, I feel that I use something old and poor. When I define anything with TypeScript it feels like something great. Think about Mercedes vs very old Hyundai Getts...

[spacether]

Payloads are defined with json schemas not openapi keywords (ignoring discriminator) You can refer to component schemas like so:
But you cannot refer to subtypes like Partial does.

Account:
allOf:
- ref to component schema that only has what you want 
properties:
  MissingProperty:
    type: string

Please read how to use json schema. You are judging it without knowing it.

[razb-viola]

spacether, I know that it's not possible thats why I wrote this as my user feedback for the next version!

[MikeRalphson]

I was talking a bit with JSON Schema maintainers long ago about difficulties related to generating a code using the specification, and they said there is a project focused on this problem but it lacked human resources at that moment. I believe you can hop into their Slack and ask for some details about this, and probably offer some help, but it won't be as easy as just adding some TypeScript-related keywords into the spec.

https://github.com/json-schema-org/vocab-idl

[spacether]

How about requiring the same naming convention that protobuf uses for components and property names? https://protobuf.dev/programming-guides/style/
That would make generating class and property names easier.

[ralfhandl]

The referenced protobuf style guide says

Note that protocol buffer style has evolved over time

This kind of evolution is hard or impossible if a certain style is required by a specification, and only possible with an optional style guide that accompanies a specification.

[handrews]

@earth2marsh @darrelmiller @lornajane should we transfer this discussion to the Moonwalk repo to keep things in one place? (I don't have the permissions on the moonwalk repo to do that, so please transfer if you think it should be done).

[kevinswiber]

I wish these were all separate discussions. It's hard to track them all here.

[handrews]

@kevinswiber I've gone through and marked the threads that I know have their own discussion(s), although I probably have missed some. If you (or anyone) want to split some of the other threads out into their own discussions, please feel free!

We could close this if we think we've captured everything somewhere else - I tried to note where i had not actually checked if the other discussion covers everything, so we should do that before closing.