Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updating the api chunking proposal to add remainingItemCount #3710

Closed
wants to merge 3 commits into from
Closed

Updating the api chunking proposal to add remainingItemCount #3710

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
58aa279
Updating the api chunking proposal to add remainingItemCount
May 13, 2019
5e426ff
addressing comments
May 14, 2019
adfa4fe
alpha feature and feature flag
May 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions contributors/design-proposals/api-machinery/api-chunking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,16 @@ Expose a simple chunking mechanism to allow large API responses to be broken int
```
GET /api/v1/pods?limit=500
{
"metadata": {"continue": "ABC...", "resourceVersion": "147"},
"metadata": {"continue": "ABC...", "resourceVersion": "147",
"remaingingItemCount": 950},
"items": [
// no more than 500 items
]
}
GET /api/v1/pods?limit=500&continue=ABC...
{
"metadata": {"continue": "DEF...", "resourceVersion": "147"},
"metadata": {"continue": "DEF...", "resourceVersion": "147",
"remainingItemCount": 450},
"items": [
// no more than 500 items
]
Expand Down Expand Up @@ -78,6 +80,11 @@ The server **may** limit the amount of time a continue token is valid for. Clien

The server **must** support `continue` tokens that are valid across multiple API servers. The server **must** support a mechanism for rolling restart such that continue tokens are valid after one or all API servers have been restarted.

The `remainingItemCount` returned by the server is the number of subsequent items in the list which are not included in this list response. If the list request contained label or field selectors, then the number of remaining items is unknown and this field will be unset and omitted during serialization. If the list is complete (either because the list request is not a chunking one or because this is the last chunk), then there are no more remaining items and this field is unset and is omitted during serialization. Servers older than v1.15 omit this field in their response to any list request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it required to be exact? What’s the use case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when I’m halfway through a list?


For example, assuming there are 1450 pods in the cluster when the client sends a chunking list request for pods, with `limit` set to 500. The first chunked list response contains 500 pods, a `continue` token, and with `metadata.remainingItemCount` set to 950. If the client use the `continue` token to continue listing, the server returns the second chunked list response containing the next 500 pods, another `continue` token, and with `metadata.remainingItemCount` set to 450. Note that the `remainingItemCount` is calculated based on the consistent list taken at the first chunking list request, that is, no matter if pods are created or deleted between the first and the second chunking list requests, the `metadata.remainingItemCount` in the second list response is always set to 450.

The `remainingItemCount` offers a simple and efficient way to get the count of objects of a resource type. For example, to get the total count of all pods in the cluster, simply sends `GET /api/v1/pods?limit=1` to the apiserver. Similarly, one can get the total count of all pods in a namespace. Note that the `remainingItemCount` is set to 0 when the list request contains any label or field selector, so this feature cannot be used to count the number of objects matching specific label or field selectors. Without the `remainingItemCount` feature, one needs to list all pods to get a count.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since old servers do not support this, and I don't think we can require this value in all possible future implementations, you need to specifically describe to a client how they can tell when the server doesn't support this field, and instruct them how to correctly detect it (as other sections do).

Effectively the algorithm is:

  1. Make a limit request
  2. Observe whether continue is set (if not, you have only one or zero items)
  3. Observe whether remainingItemCount is set (if not, remainingItemCount isn't supported)


### Proposed Implementations

Expand All @@ -91,6 +98,8 @@ Implementations that cannot offer consistent ranging (returning a set of results

For etcd3 the continue token would contain a resource version (the snapshot that we are reading that is consistent across the entire LIST) and the start key for the next set of results. Upon receiving a valid continue token the apiserver would instruct etcd3 to retrieve the set of results at a given resource version, beginning at the provided start key, limited by the maximum number of requests provided by the continue token (or optionally, by a different limit specified by the client). If more results remain after reading up to the limit, the storage should calculate a continue token that would begin at the next possible key, and the continue token set on the returned list.

etcd3 returns the total number of keys within the range as `response.Count` if the read is a range read. The storage checks if the list request contained label or field selector. If not, the storage calculates the `remainingItemCount` as `response.Count-limit`, which is the number of remaining items in the consistent list, excluding the items in the current and previous chunked list response.

The storage layer in the apiserver must apply consistency checking to the provided continue token to ensure that malicious users cannot trick the server into serving results outside of its range. The storage layer must perform defensive checking on the provided value, check for path traversal attacks, and have stable versioning for the continue token.

#### Possible SQL database implementation
Expand Down