Best Practices for Browsable APIs

[matthewhanson]

As I was cleaning up the API Spec, I came across this:

---

An additional best practice is to use the WFS items available in /collections/{collectionId}/items as the "canonical" web location. Then the STAC Catalogs returned from /stac/ can either link directly to those (from the appropriate sub-catalog - for example /stac/landsat8/42/31/2017/ would be a catalog consisting of links to /collections/). Or it can return JSON in the link structure (like /stac/landsat8/42/31/2017/item203123.json), and have that returned JSON use a link with rel=canonical that goes back to the Item that is in the collection.

---

I removed it, but not because I think it without merit, but it's just not enough information for an implementer, and it's really not part of the spec. We've talked about this before, and the idea is basically that partial URL components that don't direct to an actual Item will return a STAC catalog result, mirroring what we see with static catalogs. For instance, http://landsat-stac.s3.amazonaws.com/landsat-8-l1/151/018/catalog.json isn't an Item, or the root catalog, or a collection, it is a sub-catalog, and if we were to visit https://sat-api.developmentseed.org/collections/landsat-8-l1/151/018 it would be nice to have it also return a similar sub-catalog so that one can "browse" the catalog.

@joshfix implemented something like this, but dynamically. Whereas the above represents a singular pre-defined view into the data,with dynamically generated "views" one could browse the API using any STAC metadata field. With the above one browses by path, then by row, then by date, but this would allow you to browse by e.g. Cloud cover and then sensor.

Currently, the STAC API however does not define what a sub-catalog is, nor does the spec define an Item being a child of a Catalog (or Collection). Instead Collections have a link to rel=Items (not rel=Item!!), which is a search that returns all items within that collection.

To do a browsable API correctly though, I think an API would also need to implement aggregations somehow. If you are browsing through metadata you will need to generate links dynamically of all the possible values for the specified metadata field. I think this means needing to define aggregation endpoints in order to support it, and they would be useful on their own anyway.

So I removed this paragraph because a browsable API warrants it's own discussion and more than a few sentences in the spec.

[cholmes]

+1 - yeah, sorry for not raising it on its own, thanks for separating out here.

Scheduling for 0.8.0, and added a new 'stac sprint 4 discuss' label to mark up those that we should talk about in person.

[cholmes]

Moving this to beta.2. @matthewhanson and @m-mohr and I discussed this on a phone call 11-16-20, and agreed that with beta.1 we'll likely have the core spec in much better place for 'browse' API's, since we're aiming to make it very much discoverable / browsable, and the constructs with STAC core should be compatible.

We talked about just closing this if we did that, but I brought up that it'd be good to talk in depth about some best practices of how to organize the browsability in a useful way.

[m-mohr]

Yes, and later I realized that your way of defining browsable is a bit different than mine. Maybe it should be clarified what browsable actually means here... For me browsable was always just having links so that people can browse through catalogs without having to follow the OGC API endpoints. But it seems the actual idea was somehow a simpler search with dynamic catalogs based on some filter parameters or so?!

[matthewhanson]

That's the same way I think of browsing...it's just links providing a structure to drill down through the catalog just like with a static catalog.

However, with an API, since you'd create links dynamically anyway you can just have a string template that describes what that structure would be.

I got this implemented and working in cmr-stac this week and it works with STAC browser (will get to production in a couple weeks). It currently just supports temporal. I define the template as "year/month/day", then the API creates those links so you can browse like:

/collections/<colid>/<year>/<month>/<day>

So you could have it sorta act like a search if you allowed a user to specify what that path was. In the case of cmr-stac I did not do that and it's just set as an envvar for all collections. But I could picture someone wanting to have different browse structures (views) for different collections, or even allow a user to specify how they want to browse. One user may want to browse by tile location first and then date, whereas someone else is interested in global data so just wants to browse by date.

This would allow someone to easily crawl through a subtree of their own creation.

[philvarner]

I'm going to bump this out from beta.2, since it doesn't appear that the spec work for it has started and it's not a critical piece right now.