Add an API concepts document and describe terminology and API chunking #6540

smarterclayton · 2017-12-02T21:07:10Z

Create a new API reference page that covers some high level topics

terminology, paths, verbs, watching resources, chunking, and content
type negotation.

Needed a place to define chunking, and we don't have a doc for it.

@kubernetes/sig-api-machinery-api-reviews

Covers kubernetes/enhancements#365

This change is

smarterclayton · 2017-12-02T21:08:09Z

@kubernetes/api-reviewers a new high level API doc for the public docs so I had someplace to document chunking.

k8sio-netlify-preview-bot · 2017-12-02T21:08:58Z

Deploy preview for kubernetes-io-master-staging ready!

Built with commit 1c7d896

https://deploy-preview-6540--kubernetes-io-master-staging.netlify.com

justinsb

Minor nits / feedback, but LGTM

justinsb · 2017-12-02T21:40:10Z

docs/reference/api-concepts.md

+* A list of instances of a resource type is known as a **collection**
+* A single instance of the resource type is called a **resource**
+
+All resource types are either scoped by the cluster (`/apis/GROUP/VERSION/*`) or to a namespace (`/apis/GROUP/VERSION/NAMESPACE/*`). A namespace-scoped resource type will be deleted when its namespace is deleted and access to that resource type is controlled by authorization checks on the namespace scope. The following paths are used to retrieve collections and resources:


namespaces/<namespace>/, I think?

justinsb · 2017-12-02T21:48:05Z

docs/reference/api-concepts.md

+          "items": [...] // returns pods 1001-1253
+        }
+
+Note that the `resourceVersion` of the list remains constant across each request, indicating the server is showing us a consistent snapshot of the pods. Pods that are created, updated, or deleted after version `10245` would not be shown unless the user makes a list request without the `continue` token.  This allows clients to break large requests into smaller chunks and t hen perform a watch operation on the full set without missing any updates.


Typo: extra space in then ("and t hen perform a watch operation")

justinsb · 2017-12-02T21:53:44Z

docs/reference/api-concepts.md

+
+### Protobuf encoding
+
+Kubernetes uses an envelope wrapper to encode Protobuf responses, since unlike JSON a Protobuf message is not self-describing (the client cannot determine the type).


The point you're trying to make here didn't immediately click with me.

Out of interest, why didn't we just require that TypeMeta always be field 1?

It is by convention today, but we wanted to make it easy for a client to sniff content type as well for things on disk or in etcd. Proto doesn't require field 1 to be at the beginning of a serialized object (at least, pretty sure it doesn't) so we wouldn't have been able to sniff for it.

Oh, we also wanted to support gzipping in the future.

justinsb · 2017-12-02T21:55:16Z

docs/reference/api-concepts.md

+```
+
+Clients that receive a response in `application/vnd.kubernetes.protobuf` that does not match the expected prefix should reject the response since
+future versions may need to alter the serialization format in an incompatible way, and will do so by changing the prefix.


I'd hope we define a new MIME type instead!

Wouldn't help on disk, unfortunately.

tengqm · 2017-12-03T01:03:15Z

docs/reference/api-concepts.md

+  * `GET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE` - return collection of all instances of the resource type in NAMESPACE
+  * `GET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE/NAME` - return the instance of the resource type with NAME in NAMESPACE
+
+Since a namespace is a cluster-scoped resource type, you can retrieve the list of all namespaces with `GET /api/v1/namespaces` and details about a particular namespace with `GET /api/v1/namespaces/name`.


GET /api/v1/namespaces/name

GET /api/v1/namespaces/NAME

tengqm · 2017-12-03T01:10:17Z

docs/reference/api-concepts.md

+
+On large clusters, retrieving the collection of some resource types may result in very large responses that can impact the server and client. For instance, a cluster may have tens of thousands of pods, each of which is 1-2kb of encoded JSON. Retrieving all pods across all namespaces may result in a very large response (10-20MB) and consume a large amount of server resources. Starting in Kubernetes 1.9 the server supports the ability to break a single large collection request into many smaller chunks while preserving the consistency of the total request. Each chunk can be returned sequentially which reduces both the total size of the request and allows user-oriented clients to display results incrementally to improve responsiveness.
+
+To retrieve a single list in chunks, two new parameters `limit` and `continue` are supported on collection requestns and a new field `continue` is returned from all list operations in the list `metadata` field. A client should specify the maximum results they wish to receive in each chunk with `limit` and the server will return up to `limit` resources in the result and include a `continue` value if there are more resources in the collection. The client can then pass this `continue` value to the server on the next request to instruct the server to return the next chunk of results. By continuing until the server returns an empty `continue` value the client can consume the full set of results. 


supported on collection requestns

supported on collection requests

Create a new API reference page that covers some high level topics - terminology, paths, verbs, watching resources, chunking, and content type negotation.

smarterclayton · 2017-12-13T18:30:24Z

Updated with nits fixed.

smarterclayton · 2017-12-19T15:58:46Z

Any other reviewers have comments?

cben · 2018-01-10T17:07:39Z

docs/reference/api-concepts.md

+
+## Efficient detection of changes
+
+To enable clients to build a model of the current state of a cluster, all Kubernetes object resource types are required to support consistent lists and an incremental change notification feed called a **watch**.  Every Kubernetes object has a `resourceVersion` field representing the version of that resource as stored in the underlying database. When retrieving a collection of resources (either namespace or cluster scoped), the response from the server will contain a `resourceVersion` value that can be used to initiate a watch against the server. The server will return all changes (creates, deletes, and updates) that occur after the supplied `resourceVersion`. This allows a client to fetch the current state and then watch for changes without missing any updates. If the client watch is disconnected they can restart a new watch from the last returned `resourceVersion`, or perform a new collection request and begin again.


restart a new watch from the last returned resourceVersion

Ooh, is this official? 📜 👑
We've bumped into how to restart watch from same point in ruby kubeclient (ManageIQ/kubeclient#275) and I see python client too (kubernetes-client/python#124)...

So far all I read said resourceVersion is opaque and it wasn't clear they come from same timeline for different objects and their collection...
Unlike List that returns both collection resourceVersion and individual objects' versions, during watch you only see updates to individual objects' versions.

I see experimentally that the collection's resourceVersion as returned by List is de-facto same for all collections, increments on any change to object in any collection, and equals resourceVersion of that last changed object, so this would work.

So, can clients assume collection resourceVersion >= max(obj.resourceVersion for obj in collection)?
Can clients assume watching collection from max(obj.resourceVersion seen during previous watch) will yield same point watch stopped? (as long as that history is not lost...)

More things that would be great if docs explicitly allowed or disallowed:

Can clients take a collection version (from List) and use it on a single resource watch?

Can clients take version from one collection and use it on another collection?

Can clients do any kind of "greater than" comparisons between versions?
Or only know that last string seen, over single watch, is semantically latest?

Ooh, is this official?

Yes.

We've bumped into how to restart watch from same point

You can always attempt to restart from the last resourceVersion you received. If the api server no longer has enough history to let you start from that point, it will return a 410 error and you'll have to relist to get a new fresh resourceVersion

So far all I read said resourceVersion is opaque and it wasn't clear they come from same timeline for different objects and their collection...

That is correct. You should not make any assumptions about how two resourceVersions relate to each other.

I see experimentally that the collection's resourceVersion as returned by List is de-facto same for all collections, increments on any change to object in any collection, and equals resourceVersion of that last changed object, so this would work.

Things you can do:

Remember the last resourceVersion you received for a resource, and use it when re-establishing a watch for that resource

Compare the resourceVersion for an object retrieved at two different times to see if it is identical

Things you cannot do:

Assume the resourceVersion is numeric

Assume you can derive meaning from two different resourceVersions (e.g. "bigger number came later")

Compare resourceVersions between two resources, or between two resource types

So, can clients assume collection resourceVersion >= max(obj.resourceVersion for obj in collection)?

No

Can clients assume watching collection from max(obj.resourceVersion seen during previous watch) will yield same point watch stopped? (as long as that history is not lost...)

You should not take the max... that involves interpreting the resourceVersion. You should treat it as opaque and remember the most recent one received

@smarterclayton maybe you can incorporate some of the below, or do we have a canonical place for this to be documented to link to?

@cben, answers below.

So, can clients assume collection resourceVersion >= max(obj.resourceVersion for obj in collection)?

Per collection, this happens to be true today. However the only operation on resourceVersions that we're promising will work in the indefinite future is that of equality (==). One can imagine data storage techniques where resourceVersions aren't linear or aren't numeric. Please don't bake assumptions about resourceVersions into clients, that's not future proof.

Can clients assume watching collection from max(obj.resourceVersion seen during previous watch) will yield same point watch stopped? (as long as that history is not lost...)

Do not use max; use last seen. Then, yes, that is the way it is intended to function.

Can clients take a collection version (from List) and use it on a single resource watch?

Allowed.

Can clients take version from one collection and use it on another collection?

NO. Empirically it works... until it doesn't. The cluster administrator can choose to move resource types between different backends, which would cause this to stop being true. The default setup for Events already does this.

Can clients do any kind of "greater than" comparisons between versions?

NO.

Or only know that last string seen, over single watch, is semantically latest?

Probably not safe depending on what you want to do, in that there could be things you haven't yet seen over the watch.

Thanks! <3

Things you can do:

Remember the last resourceVersion you received for a resource, and use it when re-establishing a watch for that resource

Right, for watch on single resource this was always clear.

Just to emphasize, I'm asking about watch of a collection, and restarting it from the last version of an individual resource within that collection — which is all you get as you watch.

@lavalamp hello, a quick question: so today, we can say for EntityA, if it has bigger resourceVersion, it is newer (due to etcd's modifiedIndex).

So for the future, will there always be some way to say which version of the EntityA is newer? Because we should have such attribute. I would guess that many applications have this need.

today, we can say for EntityA, if it has bigger resourceVersion, it is newer (due to etcd's modifiedIndex).

that is not guaranteed by the API

will there always be some way to say which version of the EntityA is newer?

there is creationTimestamp on the object, but other than that, no.

@liggitt would it be possible to add something like that? It's quite common e.g. for public clouds, that entity has something comparable (e.g. updated_on timestamp)

So I wonder why we don't have it here? Is there something blocking it? Or is it just because nobody had the usecase?

Our usecase is we want to process the data in parallel, without something comparable, we are forced to do everything in 1 process (which is quite bad when we have envs with 100k of containers, or other entities)

But even for single process, if we combine the data from API and watches, the data from watches can temporarily give us old data, that we want to throw away, but we can't because we don't know what is newer.

if we combine the data from API and watches, the data from watches can temporarily give us old data, that we want to throw away, but we can't because we don't know what is newer.

combining data from two streams that differ in time seems problematic. Typically, controllers are driven by watch alone, and if updates to the API hit a conflict error, that means the resource was updated in the meantime, and the controller can simply wait for the next event to arrive via the watch stream

we want to process the data in parallel, without something comparable, we are forced to do everything in 1 process

the workqueue used by most of the kubernetes controllers allows parallel processing of watch events... seeing how that is structured might be helpful (https://github.com/kubernetes/client-go/blob/master/examples/workqueue/main.go#L131-L133)

@liggitt Thank you, I'll try to check it out, although my Go skills are not great :-)

In detail what we want to do:

We fetch and save k8s data for various purposes (reporting, chargeback, etc.) into our Postgre SQL DB.

Given we have e.g. 100k pods and the app was down for a while (therefore the watches history is missing), we'd like to fetch it and start watches at the same time. So we get fresh changes immediately, while we are getting the full inventory on the background (saving in all in parallel into our DB)

Fetching 100k pods and associated objects and storing them takes quite some time, so if we need to do this sequentially, the user needs to wait to get fresh changes (in bigger envs it's like 0.5h or more of processing time). Also the sequential processing needs more orchestration and that makes it more complex.

There is no way of saving the data in parallel, if we can't compare the version of entities. So what would you advise here? I am not sure if kubernetes controllers are solving the same issue as described ^ ?

So API and watches are taking the data from different sources? So if we would put e.g. the timestamp to k8s db for pods, the watches would not see it? Only the API query? Or what is the reason we can't have a comparable attribute?

smarterclayton · 2018-01-12T20:49:46Z

I'll add some of these comments if @lavalamp promises to review afterwards so we can merge this :)

lavalamp · 2018-01-12T21:06:27Z

Walked into that one, but OK, will do. I'm OOO monday & tuesday, though.

…

On Fri, Jan 12, 2018 at 12:51 PM, Clayton Coleman ***@***.***> wrote: I'll add some of these comments if @lavalamp <https://github.com/lavalamp> promises to review afterwards so we can merge this :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6540 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAnglgQuiiPj80n0l7BjykIKONAnppdrks5tJ8XCgaJpZM4Qzd1v> .

zacharysarah

This is so awesome. ✨ Your writing is clear throughout and represents some of the best in the repo. Minor fixes only--good work.

zacharysarah · 2018-01-12T22:12:28Z

docs/reference/api-concepts.md

+          "items": [...]
+        }
+
+2. Starting from resource version 10245, receive notifications of any creates, deletes, or updates as individual JSON objects


Add a period at the end of the sentence.

zacharysarah · 2018-01-12T22:12:43Z

docs/reference/api-concepts.md

+
+For example:
+
+1. List all of the pods in a given namespace


Add a period at the end of the sentence.

zacharysarah · 2018-01-12T22:15:42Z

docs/reference/api-concepts.md

+        }
+        ...
+
+A given Kubernetes server will only preserve a historical list of changes for a limited time. On older clusters using etcd2 a maximum of 1000 changes will be preserved and on newer clusters using etcd3 changes in the last 5 minutes are preserved by default.  Clients must handle the case where the requested watch operations fails because the historical version of that resource is not available by recognizing the status code `410 Gone`, clearing their local cache, performing a list operation, and starting the watch from the `resourceVersion` returned by that new list operation. Most client libraries offer some form of standard tool for this logic (in Go this is called a `Reflector` and is located in the `k8s.io/client-go/cache` package).


Replace this paragraph with:

A given Kubernetes server will only preserve a historical list of changes for a limited time. Older clusters using etcd2 preserve a maximum of 1000 changes. Newer clusters using etcd3 preserve changes in the last 5 minutes by default. When the requested watch operations fail because the historical version of that resource is not available, clients must handle the case by recognizing the status code `410 Gone`, clearing their local cache, performing a list operation, and starting the watch from the `resourceVersion` returned by that new list operation. Most client libraries offer some form of standard tool for this logic. (In Go this is called a `Reflector` and is located in the `k8s.io/client-go/cache` package.)

So:

A given Kubernetes server will only preserve a historical list of changes for a limited time. Older clusters using etcd2 preserve a maximum of 1000 changes. Newer clusters using etcd3 preserve changes in the last 5 minutes by default. When the requested watch operations fail because the historical version of that resource is not available, clients must handle the case by recognizing the status code 410 Gone, clearing their local cache, performing a list operation, and starting the watch from the resourceVersion returned by that new list operation. Most client libraries offer some form of standard tool for this logic. (In Go this is called a Reflector and is located in the k8s.io/client-go/cache package.)

zacharysarah · 2018-01-12T22:17:05Z

docs/reference/api-concepts.md

+
+Like a watch operation, a `continue` token will expire after a short amount of time (by default 5 minutes) and return a `410 Gone` if more results cannot be returned. In this case, the client will need to start from the beginning or omit the `limit` parameter.
+
+For example, if there are 1253 pods on the cluster, and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:


For example, if there are 1,253 pods on the cluster and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:

So:

For example, if there are 1,253 pods on the cluster and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:

zacharysarah · 2018-01-12T22:17:22Z

docs/reference/api-concepts.md

+
+For example, if there are 1253 pods on the cluster, and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:
+
+1. List all of the pods on a cluster, retrieving up to 500 pods each time


Add a period at the end of the sentence.

zacharysarah · 2018-01-12T22:19:06Z

docs/reference/api-concepts.md

+
+## Alternate representations of resources
+
+By default Kubernetes returns objects serialized to JSON with content type `application/json`. This is the default serialization format for the API. However, clients may request the more efficient Protobuf representation of these objects for better performance at scale. The Kubernetes API implements standard HTTP content type negotation - passing an `Accept` header with a `GET` call will request that the server return objects in the provided content type, while sending an object in Protobuf to the server for a `PUT` or `POST` call takes the `Content-Type` header. The server will return a `Content-Type` header if the requested format is supported, or the `406 Not acceptable` error if an invalid content type is provided.


Tiny fix:

By default Kubernetes returns objects serialized to JSON with content type `application/json`. This is the default serialization format for the API. However, clients may request the more efficient Protobuf representation of these objects for better performance at scale. The Kubernetes API implements standard HTTP content type negotation: passing an `Accept` header with a `GET` call will request that the server return objects in the provided content type, while sending an object in Protobuf to the server for a `PUT` or `POST` call takes the `Content-Type` header. The server will return a `Content-Type` header if the requested format is supported, or the `406 Not acceptable` error if an invalid content type is provided.

So:

By default Kubernetes returns objects serialized to JSON with content type application/json. This is the default serialization format for the API. However, clients may request the more efficient Protobuf representation of these objects for better performance at scale. The Kubernetes API implements standard HTTP content type negotation: passing an Accept header with a GET call will request that the server return objects in the provided content type, while sending an object in Protobuf to the server for a PUT or POST call takes the Content-Type header. The server will return a Content-Type header if the requested format is supported, or the 406 Not acceptable error if an invalid content type is provided.

zacharysarah · 2018-01-12T22:19:23Z

docs/reference/api-concepts.md

+
+For example:
+
+1. List all of the pods on a cluster in Protobuf format


Add a period at the end of the sentence.

zacharysarah · 2018-01-12T22:19:29Z

docs/reference/api-concepts.md

+        Content-Type: application/vnd.kubernetes.protobuf
+        ... binary encoded PodList object
+
+2. Create a pod by sending Protobuf encoded data to the server, but request a response in JSON


Add a period at the end of the sentence.

zacharysarah · 2018-01-12T22:20:53Z

docs/reference/api-concepts.md

+  }
+```
+
+Clients that receive a response in `application/vnd.kubernetes.protobuf` that does not match the expected prefix should reject the response since


Clients that receive a response in `application/vnd.kubernetes.protobuf` that does not match the expected prefix should reject the response, as

So:

Clients that receive a response in application/vnd.kubernetes.protobuf that does not match the expected prefix should reject the response, as

zacharysarah · 2018-01-12T22:21:25Z

docs/reference/api-concepts.md

+```
+
+Clients that receive a response in `application/vnd.kubernetes.protobuf` that does not match the expected prefix should reject the response since
+future versions may need to alter the serialization format in an incompatible way, and will do so by changing the prefix.


future versions may need to alter the serialization format in an incompatible way and will do so by changing the prefix.

So:

future versions may need to alter the serialization format in an incompatible way and will do so by changing the prefix.

zacharysarah · 2018-01-25T22:24:28Z

@smarterclayton 👋 Bump for review feedback. Alternately, if you un-check and re-check the box to allow edits from maintainers, I'm happy to make the changes for you.

smarterclayton · 2018-01-25T22:27:13Z

Allowed maintainers to edit.

Adding review feedback

zacharysarah · 2018-01-25T22:53:32Z

/lgtm

k8s-ci-robot · 2018-01-25T22:53:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton, zacharysarah

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [zacharysarah]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

…henopis-user-journeys * 'master' of https://github.com/kubernetes/website: (102 commits) Change deployment group (#7112) fix typos in extending doc (#7110) added installation via Powershell Gallery (#6086) Update StatefulSet API version to 1.9 for the Cassandra example (#7096) Modify the terms by document style (#7026) now that phase out in k8s/cluster/ directory, so remove relative docs (#6951) Update mysql-wordpress-persistent-volume.md (#7080) Update high-availability.md (#7086) Feature gates reference documentation (#6364) Add link to autoscaler FAQ (#7045) Replace regular characters with HTML entities. (#7038) Remove unnecessary manual node object creation (#6765) upper case restriction doesn't exist (#7003) Add an API concepts document and describe terminology and API chunking (#6540) Add kube-apiserver, kube-controller-manager, kube-scheduler and etcd to glossary. (#6600) Update what-is-kubernetes.md (#6971) Fixed the interacting with cluster section for the ubuntu installation (#6905) Update weave-network-policy.md (#6960) Added AWS eks (#6568) Update eviction strategy to include priority (#6949) ... # Conflicts: # _data/setup.yml # _data/tutorials.yml # docs/imported/release/notes.md

kubernetes#6540) * Add an API concepts document and describe terminology and API chunking Create a new API reference page that covers some high level topics - terminology, paths, verbs, watching resources, chunking, and content type negotation. * Update api-concepts.md Adding review feedback

smarterclayton mentioned this pull request Dec 2, 2017

Support paged LIST queries from the Kubernetes API kubernetes/enhancements#365

Closed

k8s-github-robot assigned zacharysarah Dec 2, 2017

smarterclayton added this to the 1.9 milestone Dec 2, 2017

smarterclayton force-pushed the list branch from de65ef9 to 7d816b4 Compare December 2, 2017 21:14

justinsb reviewed Dec 2, 2017

View reviewed changes

tengqm reviewed Dec 3, 2017

View reviewed changes

Add an API concepts document and describe terminology and API chunking

182828c

Create a new API reference page that covers some high level topics - terminology, paths, verbs, watching resources, chunking, and content type negotation.

smarterclayton force-pushed the list branch from 7d816b4 to 182828c Compare December 13, 2017 18:30

cben reviewed Jan 10, 2018

View reviewed changes

zacharysarah suggested changes Jan 12, 2018

View reviewed changes

zacharysarah added the Docs Review: Open Issues label Jan 12, 2018

cben mentioned this pull request Jan 22, 2018

document that watcher stops see #273 and add each_with_retry ManageIQ/kubeclient#275

Open

Update api-concepts.md

1c7d896

Adding review feedback

zacharysarah approved these changes Jan 25, 2018

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 25, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 25, 2018

k8s-ci-robot merged commit 282455f into kubernetes:master Jan 25, 2018

cben mentioned this pull request Aug 22, 2018

Create OpenShift Service Catalog tables ManageIQ/manageiq-schema#260

Merged

agrare mentioned this pull request Aug 22, 2018

Restart stale watch threads where they left off ManageIQ/manageiq-providers-kubernetes#275

Merged

cben mentioned this pull request Oct 18, 2018

watching stops without any notification ManageIQ/kubeclient#273

Open


		### Protobuf encoding

		Kubernetes uses an envelope wrapper to encode Protobuf responses, since unlike JSON a Protobuf message is not self-describing (the client cannot determine the type).


		On large clusters, retrieving the collection of some resource types may result in very large responses that can impact the server and client. For instance, a cluster may have tens of thousands of pods, each of which is 1-2kb of encoded JSON. Retrieving all pods across all namespaces may result in a very large response (10-20MB) and consume a large amount of server resources. Starting in Kubernetes 1.9 the server supports the ability to break a single large collection request into many smaller chunks while preserving the consistency of the total request. Each chunk can be returned sequentially which reduces both the total size of the request and allows user-oriented clients to display results incrementally to improve responsiveness.

		To retrieve a single list in chunks, two new parameters `limit` and `continue` are supported on collection requestns and a new field `continue` is returned from all list operations in the list `metadata` field. A client should specify the maximum results they wish to receive in each chunk with `limit` and the server will return up to `limit` resources in the result and include a `continue` value if there are more resources in the collection. The client can then pass this `continue` value to the server on the next request to instruct the server to return the next chunk of results. By continuing until the server returns an empty `continue` value the client can consume the full set of results.


		## Efficient detection of changes

		To enable clients to build a model of the current state of a cluster, all Kubernetes object resource types are required to support consistent lists and an incremental change notification feed called a watch. Every Kubernetes object has a `resourceVersion` field representing the version of that resource as stored in the underlying database. When retrieving a collection of resources (either namespace or cluster scoped), the response from the server will contain a `resourceVersion` value that can be used to initiate a watch against the server. The server will return all changes (creates, deletes, and updates) that occur after the supplied `resourceVersion`. This allows a client to fetch the current state and then watch for changes without missing any updates. If the client watch is disconnected they can restart a new watch from the last returned `resourceVersion`, or perform a new collection request and begin again.


		Like a watch operation, a `continue` token will expire after a short amount of time (by default 5 minutes) and return a `410 Gone` if more results cannot be returned. In this case, the client will need to start from the beginning or omit the `limit` parameter.

		For example, if there are 1253 pods on the cluster, and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:


		For example, if there are 1253 pods on the cluster, and the client wants to receive chunks of 500 pods at a time, they would request those chunks as follows:

		1. List all of the pods on a cluster, retrieving up to 500 pods each time


		## Alternate representations of resources

		By default Kubernetes returns objects serialized to JSON with content type `application/json`. This is the default serialization format for the API. However, clients may request the more efficient Protobuf representation of these objects for better performance at scale. The Kubernetes API implements standard HTTP content type negotation - passing an `Accept` header with a `GET` call will request that the server return objects in the provided content type, while sending an object in Protobuf to the server for a `PUT` or `POST` call takes the `Content-Type` header. The server will return a `Content-Type` header if the requested format is supported, or the `406 Not acceptable` error if an invalid content type is provided.


		For example:

		1. List all of the pods on a cluster in Protobuf format

Add an API concepts document and describe terminology and API chunking #6540

Add an API concepts document and describe terminology and API chunking #6540

Conversation

smarterclayton commented Dec 2, 2017 • edited by k8s-reviewable Loading

smarterclayton commented Dec 2, 2017

k8sio-netlify-preview-bot commented Dec 2, 2017 • edited Loading

justinsb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justinsb Dec 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smarterclayton commented Dec 13, 2017

smarterclayton commented Dec 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cben Jan 10, 2018 • edited Loading

Choose a reason for hiding this comment

Ladas Aug 22, 2018 • edited Loading

Choose a reason for hiding this comment

liggitt Aug 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smarterclayton commented Jan 12, 2018

lavalamp commented Jan 12, 2018 via email

zacharysarah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zacharysarah commented Jan 25, 2018

smarterclayton commented Jan 25, 2018

zacharysarah commented Jan 25, 2018

k8s-ci-robot commented Jan 25, 2018

smarterclayton commented Dec 2, 2017 •

edited by k8s-reviewable

Loading

k8sio-netlify-preview-bot commented Dec 2, 2017 •

edited

Loading

justinsb Dec 2, 2017 •

edited

Loading

cben Jan 10, 2018 •

edited

Loading

Ladas Aug 22, 2018 •

edited

Loading

liggitt Aug 22, 2018 •

edited

Loading