Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: define paging for collections and items endpoint #221

Merged
merged 19 commits into from
Dec 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,22 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v1.0.0-beta5 - Unreleased] - TBD
## [v1.0.0-beta.5 - Unreleased] - TBD

### Added

- The paging mechanism via a Link with rel `next` or `prev` as defined for Item Search can also be used
for the STAC API - Features endpoint `/collections/{collection_id}/items`, as described in OAFeat.
- The paging mechanism via a Link with rel `next` or `prev` as defined for items can also be used
for the STAC API - Features and STAC API - Collections endpoint `/collections`.

### Changed

- Limit parameter semantics now match OAFeat. Previously, the behavior was not precisely defined.
- `service-desc` endpoint may return any service description (previous restricted to only OpenAPI 3.0 JSON)
- `service-desc` endpoint may return any service description format, typically a
machine-consumable one (previous restricted required to be OpenAPI 3.0 JSON)
- `service-doc` endpoint may return any service description format, typically a
human-consumable one (previous restricted required to be HTML)

### Deprecated

Expand Down
7 changes: 7 additions & 0 deletions collections/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
- [STAC API - Collections](#stac-api---collections)
- [Link Relations](#link-relations)
- [Endpoints](#endpoints)
- [Pagination](#pagination)
- [Example](#example)

- **OpenAPI specification:** [openapi.yaml](openapi.yaml) ([rendered version](https://api.stacspec.org/v1.0.0-beta.5/collections))
Expand Down Expand Up @@ -90,6 +91,12 @@ elsewhere. If this is done, it is recommended to include a `rel` of `canonical`
STAC API's implementing the Collections class must support HTTP GET operation at `/collections`, with the return JSON document consisting
of an array of all STAC Collections and an array of Links.

## Pagination

The `/collections` endpoint supports a pagination mechanism that aligns with pagination as described in the
OGC API - Common - Part 2: Geospatial Data specification. This is described in detail in
the [STAC - Features Collection Pagination section](../ogcapi-features/README.md#collection-pagination).

## Example

Below is a minimal example, but captures the essence. Each collection object must be a valid STAC
Expand Down
17 changes: 12 additions & 5 deletions item-search/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- [Query Examples](#query-examples)
- [Query Parameter Table](#query-parameter-table)
- [Response](#response)
- [Paging](#paging)
- [Pagination](#pagination)
- [HTTP Request Methods and Content Types](#http-request-methods-and-content-types)
- [GET](#get)
- [POST](#post)
Expand Down Expand Up @@ -164,7 +164,7 @@ The response to a request (GET or POST) to the search endpoint must always be an
FeatureCollection](https://tools.ietf.org/html/rfc7946#section-3.3) that consists entirely of STAC
[Item](../stac-spec/item-spec/item-spec.md) objects.

### Paging
### Pagination

OGC API supports paging through hypermedia links and STAC follows the same pattern for the cross collection search. For
GET requests, a link with `rel` type `next` is supplied. This link may contain any URL parameter that is necessary
Expand All @@ -178,7 +178,11 @@ parameter name is defined by the implementor and is not necessarily part of the
"links": [
{
"rel": "next",
"href": "http://api.cool-sat.com/search?page=2"
"href": "http://api.cool-sat.com/search?page=3"
},
{
"rel": "prev",
"href": "http://api.cool-sat.com/search?page=1"
}
]
}
Expand All @@ -190,12 +194,15 @@ The href may contain any arbitrary URL parameter:
- `http://api.cool-sat.com/search?next=8a35eba9c`
- `http://api.cool-sat.com/search?token=f32890a0bdb09ac3`

Implementations may also add link relations `prev`, `first`, and `last`, though these are not required and may
be infeasible to implement in some data stores.

OAFeat does not support POST requests for searches, however the STAC API spec does. Hypermedia links are not designed
for anything other than GET requests, so providing a next link for a POST search request becomes problematic. STAC has
decided to extend the `link` object to support additional fields that provide hints to the client as to how it must
decided to extend the Link object to support additional fields that provide hints to the client as to how it must
execute a subsequent request for the next page of results.

The following fields have been added to the `link` object specification for the API spec:
The following fields have been added to the Link object specification for the API spec:

| Parameter | Type | Description |
| --------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand Down
86 changes: 86 additions & 0 deletions ogcapi-features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
- [STAC API - Features](#stac-api---features)
- [Link Relations](#link-relations)
- [Endpoints](#endpoints)
- [Item Pagination](#item-pagination)
- [Collection Pagination](#collection-pagination)
- [Examples](#examples)
- [Example Landing Page for STAC API - Features](#example-landing-page-for-stac-api---features)
- [Example Collection for STAC API - Features](#example-collection-for-stac-api---features)
Expand Down Expand Up @@ -118,6 +120,90 @@ STAC API.
Implementing OAFeat enables a wider range of clients to access the API's STAC Item objects, as it is a more widely implemented
protocol than STAC.

## Item Pagination

OAFeat supports paging through hypermedia links for the Items resource
(`/collections/{collectionId}/items`). Since only GET requests are allowed for this endpoint,
it is possible
to provide a Link with `rel` type `next` and the href of the full URL of the next page of results.
This link href must contain any URL parameters that are necessary
for the implementation to understand how to provide the next page of results, e.g., the query parameters `page`, `next`,
or `token` and any additional filter parameters if given and required. For example, the links array will have an object like:

```json
"links": [
{
"rel": "next",
"href": "http://api.cool-sat.com/collections/my_collection/items?page=2"
}
]
```

The href may contain any arbitrary URL parameter, which is implementation-specific:

- `http://api.cool-sat.com/collections/my_collection/items?page=2`
- `http://api.cool-sat.com/collections/my_collection/items?next=8a35eba9c`
- `http://api.cool-sat.com/collections/my_collection/items?token=f32890a0bdb09ac3`

In addition to supporting query parameters in the URL value of the `href` field,
STAC has extended the Link object
with additional fields to support more complex HTTP requests. These include `method` to specify
an HTTP method GET or POST, `headers` to add HTTP headers in the request, and `body` with either
the entire body for the request or only the body fields that must be changed from the previous
request, as determined by the `merge` field. These fields are described in detail in the
[Item Search](../item-search/README.md#pagination) spec.

Implementations may also add link relations `prev`, `first`, and `last`. Most API implementations
should be able to support `first`, though `last` may be infeasible to implement in some data
stores.

See the [paging examples](../item-search/examples.md#paging-examples) for additional insight.

## Collection Pagination

OAFeat does not define a specific mechanism for how clients may access all collections from servers
with many
collections. STAC - Features adds support for this with pagination (similar to the Item pagination
mechanism) through hypermedia links for the Collections resource
(`/collections`). This mechanism aligns with pagination of collections in the
OGC API - Common - Part 2: Geospatial Data specification. With this, a link with
relation `next` is included in the `links` array,
and this is used to navigate to the next page of Collection objects. The specific query
parameter used for paging is implementation specific and not defined by STAC API. For example,
an implementation may take a parameter (e.g., `page`) indicating the numeric page of results, a
base64-encoded value indicating the last result returned for the current page (e.g., `search_after` as
in Elasticsearch), or a cursor token representing backend state.

In our simple example of numerical pages, the response for `page=3` would have a
`links` array containing these two Links indicating the URLs for the next (page=4) and
previous (page=2) pages:

```none
"links": [
...
{
"rel": "prev",
"href": "http://api.cool-sat.com/collections/my_collection/items?page=2"
},
{
"rel": "next",
"href": "http://api.cool-sat.com/collections/my_collection/items?page=4"
}
]
```

In addition to supporting query parameters in the URL value of the `href` field,
STAC has extended the Link object
with additional fields to support more complex HTTP requests. These include `method` to specify
an HTTP method GET or POST, `headers` to add HTTP headers in the request, and `body` with either
the entire body for the request or only the body fields that must be changed from the previous
request, as determined by the `merge` field. These fields are described in detail in the
[Item Search](../item-search/README.md#pagination) spec.

Implementations may also add link relations `prev`, `first`, and `last`. Most API implementations
should be able to support `first`, though `last` may be infeasible to implement in some data
stores.

## Examples

Note that the OAFeat endpoints *only* allow HTTP GET. HTTP POST requests are not supported. If POST is required,
Expand Down