From 1ed894e2692284155d906a5203ee9385cb742c82 Mon Sep 17 00:00:00 2001 From: David Bitner Date: Thu, 13 Jan 2022 15:19:37 -0600 Subject: [PATCH 1/2] fixes to make text examples equivalent to json examples --- fragments/filter/README.md | 366 ++++++++++++++++++------------------- 1 file changed, 182 insertions(+), 184 deletions(-) diff --git a/fragments/filter/README.md b/fragments/filter/README.md index 900d2d68..23e4af7d 100644 --- a/fragments/filter/README.md +++ b/fragments/filter/README.md @@ -1,7 +1,7 @@ # STAC API - Filter Fragment - **OpenAPI specification:** [openapi.yaml](openapi.yaml) -- **Conformance Classes:** +- **Conformance Classes:** - Filter: - Item Search Filter: - CQL2 Text: @@ -67,56 +67,56 @@ The Filter extension provides an expressive mechanism for searching based on Item attributes. -This extension references behavior defined in the +This extension references behavior defined in the [OGC API - Features - Part 3: Filtering and the Common Query Language (CQL2)](https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/cql) and [Common Query Language (CQL2) ](https://github.com/opengeospatial/ogcapi-features/blob/master/cql2/README.md) specifications. As of November 2021, these specifications are still in draft status, but rapidly converging on -finalized behavior. Several behaviors have changed since the -last published [draft](https://portal.ogc.org/files/96288), so this spec references the latest revision in the +finalized behavior. Several behaviors have changed since the +last published [draft](https://portal.ogc.org/files/96288), so this spec references the latest revision in the [OAFeat Part 3 spec's GitHub repo](https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/cql) -and [Common Query Language (CQL2)](https://github.com/opengeospatial/ogcapi-features/blob/master/cql2/README.md)). +and [Common Query Language (CQL2)](https://github.com/opengeospatial/ogcapi-features/blob/master/cql2/README.md)). Implementers should proceed with implementation, but must be aware that minor changes may be made before these specs are final. -OAFeat Part 3 CQL2 formally defines the syntax of "CQL2" as both a text format (cql2-text) as an ABNF grammar -(largely similar to the BNF grammar in the General Model for CQL) and a JSON format (cql2-json) as a JSON Schema and -OpenAPI schema. Additionally, it defines a natural +OAFeat Part 3 CQL2 formally defines the syntax of "CQL2" as both a text format (cql2-text) as an ABNF grammar +(largely similar to the BNF grammar in the General Model for CQL) and a JSON format (cql2-json) as a JSON Schema and +OpenAPI schema. Additionally, it defines a natural language description of the declarative semantics, which were never well-defined for the original CQL language. -The CQL2 Text format has had long-standing use within -geospatial software (e.g., GeoServer), is not expected to change before final. -OGC CQL2 Text has been previously described in [OGC Filter Encoding](https://www.ogc.org/standards/filter) and -[OGC Catalogue Services 3.0 - General Model](http://docs.opengeospatial.org/is/12-168r6/12-168r6.html#62) +The CQL2 Text format has had long-standing use within +geospatial software (e.g., GeoServer), is not expected to change before final. +OGC CQL2 Text has been previously described in [OGC Filter Encoding](https://www.ogc.org/standards/filter) and +[OGC Catalogue Services 3.0 - General Model](http://docs.opengeospatial.org/is/12-168r6/12-168r6.html#62) (including a BNF grammar in Annex B). The CQL2 JSON format is newly-defined and has changed significantly during the draft process, but is believed to be stable now. -It should be noted that the "CQL" referred to here is "CQL2" defined in OGC API - Features - Part 3. This is a related, but +It should be noted that the "CQL" referred to here is "CQL2" defined in OGC API - Features - Part 3. This is a related, but different language to the "classic" OGC CQL defined in the General Model. Relatedly, CQL is **not** -referencing or related two other "CQL" languages, -the [SRU (Search/Retrieve via URL) Contextual Query Language](https://www.loc.gov/standards/sru/cql/index.html) (formerly -known as Common Query Language) or the [Cassandra Query Language](https://cassandra.apache.org/doc/latest/cql/) used by the +referencing or related two other "CQL" languages, +the [SRU (Search/Retrieve via URL) Contextual Query Language](https://www.loc.gov/standards/sru/cql/index.html) (formerly +known as Common Query Language) or the [Cassandra Query Language](https://cassandra.apache.org/doc/latest/cql/) used by the Cassandra database. -## Limitations of Item Search +## Limitations of Item Search -OAFeat defines a limited set of filtering capabilities. Filtering can only be done over a single collection and -with only a single `bbox` (rectangular spatial filter) parameter and a single datetime (instant or interval) parameter. +OAFeat defines a limited set of filtering capabilities. Filtering can only be done over a single collection and +with only a single `bbox` (rectangular spatial filter) parameter and a single datetime (instant or interval) parameter. The STAC Item Search specification extends the functionality of OAFeat in a few key ways: -- It allows cross-collection filtering, whereas OAFeat only allows filtering within a single collection. +- It allows cross-collection filtering, whereas OAFeat only allows filtering within a single collection. (`collections` parameter, accepting 0 or more collections) - It allows filtering by Item ID (`ids` parameter) - It allows filtering based on a single GeoJSON Geometry, rather than only a bbox (`intersects` parameter) -However, it does not contain a formalized way to filter based on arbitrary fields in an Item. For example, there is +However, it does not contain a formalized way to filter based on arbitrary fields in an Item. For example, there is no way to express the filter "item.properties.eo:cloud_cover is less than 10". It also does not have a way to logically combine multiple spatial or temporal filters. ## Filter expressiveness -This extension expands the capabilities of Item Search and the OAFeat Items resource with -[OAFeat Part 3 CQL2](https://portal.ogc.org/files/96288) -by providing an expressive query language to construct more complex filter predicates using operators that are similar to -those provided by SQL. This extension also supports the Queryables mechanism that allows discovery of what Item fields can be used in +This extension expands the capabilities of Item Search and the OAFeat Items resource with +[OAFeat Part 3 CQL2](https://portal.ogc.org/files/96288) +by providing an expressive query language to construct more complex filter predicates using operators that are similar to +those provided by SQL. This extension also supports the Queryables mechanism that allows discovery of what Item fields can be used in predicates. CQL2 enables more expressive queries than supported by STAC API Item Search. These include: @@ -128,30 +128,30 @@ CQL2 enables more expressive queries than supported by STAC API Item Search. The ## Conformance Classes -OAFeat Part 3 CQL2 defines several conformance classes that allow implementers to create compositions of +OAFeat Part 3 CQL2 defines several conformance classes that allow implementers to create compositions of functionality that support whatever expressiveness they need. This allows implementers to incrementally support CQL -syntax, without needing to implement a huge spec all at once. Some implementers choose not to incur the cost of -implementing functionality they do not need or may not be able to implement functionality that is not supported by -their underlying datastore, e.g., Elasticsearch does not support the spatial predicates required by the +syntax, without needing to implement a huge spec all at once. Some implementers choose not to incur the cost of +implementing functionality they do not need or may not be able to implement functionality that is not supported by +their underlying datastore, e.g., Elasticsearch does not support the spatial predicates required by the Spatial Operators conformance class, only the `S_INTERSECTS` operator in the Basic Spatial Operators class. The precise decomposition of the OAFeat conformance classes is still a work in progress, but is being finalized -rapidly (see [ogcapi-features/issues/579](https://github.com/opengeospatial/ogcapi-features/issues/579)). +rapidly (see [ogcapi-features/issues/579](https://github.com/opengeospatial/ogcapi-features/issues/579)). The STAC API Filter Extension reuses the definitions and conformance classes in OAFeat CQL, adding only the Item Search Filter conformance class -(`https://api.stacspec.org/v1.0.0-beta.5/item-search#filter:item-search-filter`) to bind +(`https://api.stacspec.org/v1.0.0-beta.5/item-search#filter:item-search-filter`) to bind the CQL2 Filter behavior to the Item Search resource. The implementation **must** support these conformance classes: -- Filter (`http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter`) defines the Queryables mechanism and +- Filter (`http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter`) defines the Queryables mechanism and parameters `filter-lang`, `filter-crs`, and `filter`. -- Basic CQL2 (`http://www.opengis.net/spec/cql2/1.0/conf/basic-cql2`) defines the basic operations allowed in - the query language used for the `filter` parameter defined by Filter. This includes logical operators (`AND`, `OR`, `NOT`), +- Basic CQL2 (`http://www.opengis.net/spec/cql2/1.0/conf/basic-cql2`) defines the basic operations allowed in + the query language used for the `filter` parameter defined by Filter. This includes logical operators (`AND`, `OR`, `NOT`), comparison operators (`=`, `<>`, `<`, `<=`, `>`, `>=`), and `IS NULL`. The comparison operators are allowed against string, numeric, boolean, date, and datetime types. -- Item Search Filter (`https://api.stacspec.org/v1.0.0-beta.5/item-search#filter:item-search-filter`) binds the Filter and - Basic CQL2 conformance classes to apply to the Item Search endpoint (`/search`). This class is the correlate of the OAFeat CQL2 Features +- Item Search Filter (`https://api.stacspec.org/v1.0.0-beta.5/item-search#filter:item-search-filter`) binds the Filter and + Basic CQL2 conformance classes to apply to the Item Search endpoint (`/search`). This class is the correlate of the OAFeat CQL2 Features Filter class that binds Filter and Basic CQL2 to the Features resource (`/collections/{cid}/items`). The implementation **must** support at least one of the "CQL2 Text" or "CQL2 JSON" conformance classes that @@ -160,55 +160,55 @@ define the CQL2 format used in the filter parameter: - CQL2 Text (`http://www.opengis.net/spec/cql2/1.0/conf/cql2-text`) defines that the CQL2 Text format is supported by Item Search - CQL2 JSON (`http://www.opengis.net/spec/cql2/1.0/conf/cql2-json`) defines that the CQL2 JSON format is supported by Item Search -If both are advertised as being supported, it is only required that both be supported for GET query parameters, and that -only that CQL2 JSON be supported for POST JSON requests. It is recommended that clients use CQL2 Text in GET requests and -CQL2 JSON in POST requests. +If both are advertised as being supported, it is only required that both be supported for GET query parameters, and that +only that CQL2 JSON be supported for POST JSON requests. It is recommended that clients use CQL2 Text in GET requests and +CQL2 JSON in POST requests. For additional capabilities, the following classes can be implemented: -- Advanced Comparison Operators - (`http://www.opengis.net/spec/cql2/1.0/conf/advanced-comparison-operators`) defines the `LIKE`, +- Advanced Comparison Operators + (`http://www.opengis.net/spec/cql2/1.0/conf/advanced-comparison-operators`) defines the `LIKE`, `BETWEEN`, and `IN` operators. **Note**: this conformance class no longer requires implementing the `lower` and `upper` functions. - Basic Spatial Operators (`http://www.opengis.net/spec/cql2/1.0/conf/basic-spatial-operators`) defines the intersects operator (`S_INTERSECTS`). -- Spatial Operators +- Spatial Operators (`http://www.opengis.net/spec/cql2/1.0/conf/spatial-operators`) defines a set of operators that are part of the Dimensionally Extended Nine-intersection Model (DE-9IM) relation operators (`S_CONTAINS`, `S_CROSSES`, `S_DISJOINT`, `S_EQUALS`, `S_INTERSECTS`, `S_OVERLAPS`, `S_TOUCHES`, and `S_WITHIN`) -- Temporal Operators +- Temporal Operators (`http://www.opengis.net/spec/cql2/1.0/conf/temporal-operators`) defines several temporal operators that provide more expressivity with datetime types than the relative comparison operators in the Basic CQL2 class. - Custom Functions (`http://www.opengis.net/spec/cql2/1.0/conf/functions`) defines support for function definition and usage. -- Arithmetic Expressions: (`http://www.opengis.net/spec/cql2/1.0/conf/arithmetic`) defines +- Arithmetic Expressions: (`http://www.opengis.net/spec/cql2/1.0/conf/arithmetic`) defines support for arithmetic expressions. -- Array Operators: (`http://www.opengis.net/spec/cql2/1.0/conf/array-operators`) +- Array Operators: (`http://www.opengis.net/spec/cql2/1.0/conf/array-operators`) defines array operators (`A_EQUALS`, `A_CONTAINS`, `A_CONTAINED_BY`, and `A_OVERLAPS`). - Property-Property Comparisons: (`http://www.opengis.net/spec/cql2/1.0/conf/property-property`) allows the use of queryables (e.g., properties) in both positions of a clause, not just in the first position. This allows predicates like `property1 == property2` be expressed, whereas the Basic CQL2 conformance class only requires comparisons against right-hand-side literals. -Additionally, if an API implements the OGC API Features endpoint, it is **recommended** that the OAFeat Part 3 Filter, -Features Filter, and Basic CQL2 conformance classes be implemented, which allow use of CQL2 filters against the -OAFeat Part 1 Features endpoint (`/collections/{collectionId}/items`). Note that POST with a JSON body -to the Features resource is not supported, as POST is used by the +Additionally, if an API implements the OGC API Features endpoint, it is **recommended** that the OAFeat Part 3 Filter, +Features Filter, and Basic CQL2 conformance classes be implemented, which allow use of CQL2 filters against the +OAFeat Part 1 Features endpoint (`/collections/{collectionId}/items`). Note that POST with a JSON body +to the Features resource is not supported, as POST is used by the [Transaction Extension](../../ogcapi-features/extensions/transaction/README.md) for creating items. ## Getting Started with Implementation -It recommended that implementers start with fully implementing only a subset of functionality. A good place to start is -implementing only the Basic CQL2 conformance class of logical and comparison operators, defining a static Queryables -schema with no queryables advertised and the `additionalProperties` field set to `true`, and -only implementing CQL2 Text. Following from that can be support for -S_INTERSECTS, defining a static Queryables schema with only the basic Item properties, and -implementing CQL2 JSON. From there, other comparison operators can be implemented and a more +It recommended that implementers start with fully implementing only a subset of functionality. A good place to start is +implementing only the Basic CQL2 conformance class of logical and comparison operators, defining a static Queryables +schema with no queryables advertised and the `additionalProperties` field set to `true`, and +only implementing CQL2 Text. Following from that can be support for +S_INTERSECTS, defining a static Queryables schema with only the basic Item properties, and +implementing CQL2 JSON. From there, other comparison operators can be implemented and a more dynamic Queryables schema. -Formal definitions and grammars for CQL2 can be found here: +Formal definitions and grammars for CQL2 can be found here: -- The [OAFeat (CQL) spec](https://portal.ogc.org/files/96288) includes an ABNF for cql2-text and both JSON Schema and +- The [OAFeat (CQL) spec](https://portal.ogc.org/files/96288) includes an ABNF for cql2-text and both JSON Schema and OpenAPI specifications for cql2-json. The standalone files are: - [cql.bnf](https://github.com/opengeospatial/ogcapi-features/blob/master/extensions/cql/standard/schema/cql.bnf) - [cql.json](https://github.com/opengeospatial/ogcapi-features/blob/master/extensions/cql/standard/schema/cql.json) @@ -219,37 +219,37 @@ Formal definitions and grammars for CQL2 can be found here: These projects have or are developing CQL or CQL2 support: -- [pygeofilter](https://github.com/geopython/pygeofilter) has support for the older ECQL standard +- [pygeofilter](https://github.com/geopython/pygeofilter) has support for the older ECQL standard (similar to CQL2 Text) and will soon have support for OGC API Part 3 CQL2 -- [GeoPython PyCQL](https://github.com/geopython/pycql/tree/master/pycql) (discontinued), and the +- [GeoPython PyCQL](https://github.com/geopython/pycql/tree/master/pycql) (discontinued), and the [Bitner fork](https://github.com/bitner/pycql) to be used in stac-fastapi - [Franklin](https://github.com/azavea/franklin) is working on it in [this PR](https://github.com/azavea/franklin/pull/750). - [Geotools](https://github.com/geotools/geotools) has support for [CQL2 text](https://github.com/geotools/geotools/tree/main/modules/library/cql/src/main/java/org/geotools/filter/text/cql2) -Note that the [xbib CQL library (JVM)](https://github.com/xbib/cql) is the OASIS Contextual Query Language, not +Note that the [xbib CQL library (JVM)](https://github.com/xbib/cql) is the OASIS Contextual Query Language, not OGC CQL, and should not be used to implement this extension, as they are significantly different query languages. -[Stacatto](https://github.com/planetlabs/staccato) uses this for their query language implementation, but it is +[Stacatto](https://github.com/planetlabs/staccato) uses this for their query language implementation, but it is not compliant with this extension. ## Queryables The Queryables mechanism allows a client to discover what terms are available for use when writing filter -expressions. These terms can be defined per-collection, and the intersection of these terms over all collections is what -is available for filtering when there are no collection restrictions. By default, these queryables are the only terms that may be used -in filter expressions, and if any term is used in expression that is not defined as a queryable and error must be -returned according to OAFeat Part 3. It is recognized that this is a severe restriction in STAC APIs that have highly variable +expressions. These terms can be defined per-collection, and the intersection of these terms over all collections is what +is available for filtering when there are no collection restrictions. By default, these queryables are the only terms that may be used +in filter expressions, and if any term is used in expression that is not defined as a queryable and error must be +returned according to OAFeat Part 3. It is recognized that this is a severe restriction in STAC APIs that have highly variable and dynamic content, so this behavior may be modified by setting the `additionalProperties` attribute in the queryables definition to `true`. As such, any syntactically-valid term for a property will be accepted, and the matching semantics are such that, if an Item does not have an attribute by that name, the value is assumed to be `null`. It is recommended to use fully-qualified property names (e.g., `properties.eo:cloud_cover`). -Queryables are advertised via a JSON Schema document retrieved from the `/queryables` endpoint. This endpoint at the root -retrieves queryables that apply to all collections. When used as a subresource of the collection resource -(e.g. /collections/collection1/queryables), it returns queryables pertaining only to that single collection. +Queryables are advertised via a JSON Schema document retrieved from the `/queryables` endpoint. This endpoint at the root +retrieves queryables that apply to all collections. When used as a subresource of the collection resource +(e.g. /collections/collection1/queryables), it returns queryables pertaining only to that single collection. -It is required to implement both of these endpoints, but for a STAC API, this may simply be a static document of the +It is required to implement both of these endpoints, but for a STAC API, this may simply be a static document of the STAC-specific fields. A basic Queryables -definitions for STAC Items should include at least the fields id, collection, geometry, and datetime. +definitions for STAC Items should include at least the fields id, collection, geometry, and datetime. ```json { @@ -280,39 +280,39 @@ definitions for STAC Items should include at least the fields id, collection, ge } ``` -Fields in Item Properties should be exposed with their un-prefixed names, and not require expressions to prefix them -with `properties`, e.g., `eo:cloud_cover` instead of `properties.eo:cloud_cover`. +Fields in Item Properties should be exposed with their un-prefixed names, and not require expressions to prefix them +with `properties`, e.g., `eo:cloud_cover` instead of `properties.eo:cloud_cover`. -There may also be support for finding what queryables are available for a subset of collections, e.g., -`/queryables?collections=c1,c3`) as described in [this issue](https://github.com/opengeospatial/ogcapi-features/issues/576). +There may also be support for finding what queryables are available for a subset of collections, e.g., +`/queryables?collections=c1,c3`) as described in [this issue](https://github.com/opengeospatial/ogcapi-features/issues/576). -The Landing Page endpoint (`/`) will have a Link with rel `http://www.opengis.net/def/rel/ogc/1.0/queryables` with an href to -the endpoint `/queryables`. Additionally, each Collection resource will have a Link to the queryables resource for that -collection, e.g., `/collections/collection1/queryables`. +The Landing Page endpoint (`/`) will have a Link with rel `http://www.opengis.net/def/rel/ogc/1.0/queryables` with an href to +the endpoint `/queryables`. Additionally, each Collection resource will have a Link to the queryables resource for that +collection, e.g., `/collections/collection1/queryables`. -The queryables endpoint returns a JSON Schema describing the names and types of terms that may be used in filter expressions. -This response is defined in JSON Schema because it is a well-specified typed schema, but it is not used for validating a JSON +The queryables endpoint returns a JSON Schema describing the names and types of terms that may be used in filter expressions. +This response is defined in JSON Schema because it is a well-specified typed schema, but it is not used for validating a JSON document derived from it. This schema defines the terms that may be used in a CQL2 filter. -These queryable terms are mapped by the service to filter Items. For example, the service may define a queryable with the -name "eo:cloud_cover" that can be used in a CQL2 expression like `eo:cloud_cover <= 10`, with the semantics that only Items where the -`properties.eo:cloud_cover` value was <= 10 would match the filter. The server would then translate this into an appropriate +These queryable terms are mapped by the service to filter Items. For example, the service may define a queryable with the +name "eo:cloud_cover" that can be used in a CQL2 expression like `eo:cloud_cover <= 10`, with the semantics that only Items where the +`properties.eo:cloud_cover` value was <= 10 would match the filter. The server would then translate this into an appropriate query for the data within its datastore. -Queryables can be static or dynamically derived. For example, `cloud_cover` might be specified to have a value 0 to 100 or a field -may have a set of enumerated values dynamically determined by an aggregation at runtime. This schema can be used by a UI or -interactive client to dynamically expose to the user the fields that are available for filtering, and provide a precise group +Queryables can be static or dynamically derived. For example, `cloud_cover` might be specified to have a value 0 to 100 or a field +may have a set of enumerated values dynamically determined by an aggregation at runtime. This schema can be used by a UI or +interactive client to dynamically expose to the user the fields that are available for filtering, and provide a precise group of options for the values of these terms. -Queryables can also be used to advertise "synthesized" property values. The only requirement in CQL2 is that the property have a type -and evaluate to literal value of that type or NULL. For example, a filter like "Items must have an Asset with an eo:band with -the common_name of 'nir'" can be expressed. A Queryable `assets_bands` could be defined to have a type of array of string and -have the semantics that it contains all of `common_name` values across all assets and bands for an Item. This could then be -filtered with the CQL2 expression `'nir' in assets_bands`. Implementations would then expand this expression into the -appropriate query against its datastore. (TBD if this will actually work or not. This is also related to the upcoming +Queryables can also be used to advertise "synthesized" property values. The only requirement in CQL2 is that the property have a type +and evaluate to literal value of that type or NULL. For example, a filter like "Items must have an Asset with an eo:band with +the common_name of 'nir'" can be expressed. A Queryable `assets_bands` could be defined to have a type of array of string and +have the semantics that it contains all of `common_name` values across all assets and bands for an Item. This could then be +filtered with the CQL2 expression `'nir' in assets_bands`. Implementations would then expand this expression into the +appropriate query against its datastore. (TBD if this will actually work or not. This is also related to the upcoming restriction on property/literal comparisons) -An implementation may also choose not to advertise any queryables, and provide the user with out-of-band information or +An implementation may also choose not to advertise any queryables, and provide the user with out-of-band information or simply let them try querying against fields. While this is not allowed according to the OGC CQL2 Queryable spec, it is allowed in STAC API by the Filter Extension. In this case, the queryables endpoint (`/queryables`) would return this document: @@ -334,18 +334,18 @@ in STAC API by the Filter Extension. In this case, the queryables endpoint (`/qu This extension adds three GET query parameters or POST JSON fields to an Item Search request: - filter-lang:`cql2-text` or `cql2-json`. If undefined, defaults to `cql2-text` for a GET request and `cql2-json` for a POST request. -- filter-crs: recommended to not be passed, but server must only accept `http://www.opengis.net/def/crs/OGC/1.3/CRS84` as +- filter-crs: recommended to not be passed, but server must only accept `http://www.opengis.net/def/crs/OGC/1.3/CRS84` as a valid value, may reject any others - filter: CQL2 filter expression API implementations advertise which `filter-lang` values are supported via conformance classes in the Landing Page. -At least one must be implemented, but it is recommended to implement both. If both are advertised as conformance classes, the -server should process either for a GET request, but may only process cql2-json for a POST request. If POST of cql2-text is not +At least one must be implemented, but it is recommended to implement both. If both are advertised as conformance classes, the +server should process either for a GET request, but may only process cql2-json for a POST request. If POST of cql2-text is not supported, the server must return a 400 error if `filter-lang=cql2-text`. ## Interaction with Endpoints -In an implementation that supports several operator classes, the Landing Page (`/`) must return a document including +In an implementation that supports several operator classes, the Landing Page (`/`) must return a document including at least these values: ```json @@ -355,7 +355,7 @@ at least these values: "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core", "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30", "http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson", - + "http://www.opengis.net/spec/ogcapi_common-2/1.0/conf/collections", "http://api.stacspec.org/v1.0.0-beta.5/core", @@ -429,26 +429,26 @@ The Queryables endpoint (`/queryables`) returns something like the following: }, "assets_bands" : { "description" : "Asset eo:bands common names", - "$ref": "https://stac-extensions.github.io/eo/v1.0.0/schema.json#/properties/eo:bands/common_name" + "$ref": "https://stac-extensions.github.io/eo/v1.0.0/schema.json#/properties/eo:bands/common_name" } }, "additionalProperties": true } ``` -Alternately, the client could retrieve the queryables for a single collection with +Alternately, the client could retrieve the queryables for a single collection with `/collections/collections1/queryables`, which may have more queryables than available for the entire catalog, since -there may be queryables that are only relevant to that collection. +there may be queryables that are only relevant to that collection. -Notice in this schema that instead of directly defining the type information about each field, we have instead used the JSON +Notice in this schema that instead of directly defining the type information about each field, we have instead used the JSON Schema `$ref` mechanism to refer to a STAC schema definition. This not only allows the reuse of these JSON Schema definitions, -but also binds an arbitrarily-named Queryable to a specific STAC field. For example, in the above we know that the -`eo:cloud_cover` field is referring to the field of the same name in the EO Extension not because they happen to have the same -name, but rather because the `$ref` indicates it. The field could just as well be named "cloud_cover", "CloudCover", or "cc", -and we would still know it filtered on the EO extension `eo:cloud_cover` field. For example, if the queryable was named +but also binds an arbitrarily-named Queryable to a specific STAC field. For example, in the above we know that the +`eo:cloud_cover` field is referring to the field of the same name in the EO Extension not because they happen to have the same +name, but rather because the `$ref` indicates it. The field could just as well be named "cloud_cover", "CloudCover", or "cc", +and we would still know it filtered on the EO extension `eo:cloud_cover` field. For example, if the queryable was named "CloudCover", a CQL2 expression using that queryable would look like `CloudCover <= 10`. -While these do seem quite complex to write and understand, keep in mind that query construction will likely be done with a +While these do seem quite complex to write and understand, keep in mind that query construction will likely be done with a more ergonomic SDK, and query parsing will be done with the help of a ABNF grammar and OpenAPI schema. These parameters/fields must be supported by the STAC Item Search endpoint and OAFeat Features resource (`/collections/$collectionId/items`). @@ -457,11 +457,11 @@ These parameters/fields must be supported by the STAC Item Search endpoint and O Note: the GET examples with query parameters are unescaped to make them easier to read. -The GET examples are assuming a call to `GET /search` and the POST examples are assuming a +The GET examples are assuming a call to `GET /search` and the POST examples are assuming a call to `POST /search`. -The parameter `filter-crs` always defaults to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API, so is not shown -in any of these examples. +The parameter `filter-crs` always defaults to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API, so is not shown +in any of these examples. ### Example 1 @@ -469,7 +469,7 @@ This example uses the queryables definition in (Interaction with Endpoints)(#int #### Example 1: GET with cql2-text -Note that `filter-lang` defaults to `cql2-text` in this case. The parameter `filter-crs` defaults +Note that `filter-lang` defaults to `cql2-text` in this case. The parameter `filter-crs` defaults to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API. ```http @@ -478,7 +478,7 @@ filter=id='LC08_L1TP_060247_20180905_20180912_01_T1_L1TP' AND collection='landsa #### Example 1: POST with cql2-json -Note that `filter-lang` defaults to `cql2-json` in this case. The parameter `filter-crs` defaults +Note that `filter-lang` defaults to `cql2-json` in this case. The parameter `filter-crs` defaults to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API. ```json @@ -486,11 +486,11 @@ to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API. "filter": { "op" : "and", "args": [ - { + { "op": "=", - "args": [ { "property": "id" }, "LC08_L1TP_060247_20180905_20180912_01_T1_L1TP" ] + "args": [ { "property": "id" }, "LC08_L1TP_060247_20180905_20180912_01_T1_L1TP" ] }, - { + { "op": "=", "args" : [ { "property": "collection" }, "landsat8_l1tp" ] } @@ -503,18 +503,16 @@ to `http://www.opengis.net/def/crs/OGC/1.3/CRS84` for a STAC API. This example uses the queryables definition in [Interaction with Endpoints](#interaction-with-endpoints). -Note that filtering on the `collection` field is relevant in Item Search, since the queries are cross-collection, whereas +Note that filtering on the `collection` field is relevant in Item Search, since the queries are cross-collection, whereas OGC API Features filters only operate against a single collection already. #### Example 2: GET with cql2-text ```http filter=collection = 'landsat8_l1tp' - AND gsd <= 30 AND eo:cloud_cover <= 10 AND datetime >= TIMESTAMP('2021-04-08T04:39:23Z') - AND datetime <= TIMESTAMP('2021-05-07T12:27:57Z') - AND S_INTERSECTS(geometry, POLYGON((43.5845 -79.5442, 43.6079 -79.4893, 43.5677 -79.4632, 43.6129 -79.3925, 43.6223 -79.3238, 43.6576 -79.3163, 43.7945 -79.1178, 43.8144 -79.1542, 43.8555 -79.1714, 43.7509 -79.6390, 43.5845 -79.5442)) + AND S_INTERSECTS(geometry, POLYGON((43.5845 -79.5442, 43.6079 -79.4893, 43.5677 -79.4632, 43.6129 -79.3925, 43.6223 -79.3238, 43.6576 -79.3163, 43.7945 -79.1178, 43.8144 -79.1542, 43.8555 -79.1714, 43.7509 -79.6390, 43.5845 -79.5442))) ``` #### Example 2: POST with cql2-json @@ -525,20 +523,20 @@ filter=collection = 'landsat8_l1tp' "filter": { "op": "and", "args": [ - { - "op": "=", - "args": [ { "property": "collection" }, "landsat8_l1tp" ] + { + "op": "=", + "args": [ { "property": "collection" }, "landsat8_l1tp" ] }, - { - "op": "<=", - "args": [ { "property": "eo:cloud_cover" }, "10" ] + { + "op": "<=", + "args": [ { "property": "eo:cloud_cover" }, 10 ] }, - { - "op": ">=", - "args": [ { "property": "datetime" }, { "timestamp": "2021-04-08T04:39:23Z" } ] + { + "op": ">=", + "args": [ { "property": "datetime" }, { "timestamp": "2021-04-08T04:39:23Z" } ] }, { - "op": "s_intersects", + "op": "s_intersects", "args": [ { "property": "geometry" @@ -570,7 +568,7 @@ filter=collection = 'landsat8_l1tp' ### Example 3: Conjunction with AND -We'll be imagining these as queries against [EarthSearch Sentinel 2 +We'll be imagining these as queries against [EarthSearch Sentinel 2 COG](https://stacindex.org/catalogs/earth-search#/Cnz1sryATwWudkxyZekxWx6356v9RmvvCcLLw79uHWJUDvt2?t=items) data. The queryables defined are as follows: @@ -610,11 +608,11 @@ The queryables defined are as follows: ``` Note that `acme:data_coverage` and `acme:grid_id` are properties that are not defined in an extension schema, and are intended to -represent vendor-specific properties. Because of this, they are fully specified directly in the JSON Schema. However, it is -recommended that vendor-specific properties be published as part of a well-defined extension schema, so these only represent ones +represent vendor-specific properties. Because of this, they are fully specified directly in the JSON Schema. However, it is +recommended that vendor-specific properties be published as part of a well-defined extension schema, so these only represent ones that have not followed that recommendation. -A sample STAC Item (excluding `assets`) is: +A sample STAC Item (excluding `assets`) is: ```json { @@ -655,15 +653,15 @@ A sample STAC Item (excluding `assets`) is: } ``` -One problem in working with Sentinel-2 data is that many scenes only contain a tiny "sliver" of data, where the satellite's -recording path intersection only a corner of a grid square. This examples shows -Show me all imagery that has low cloud cover (less than 10), and high data coverage (50), as I'd like a cloud free image that is not just +One problem in working with Sentinel-2 data is that many scenes only contain a tiny "sliver" of data, where the satellite's +recording path intersection only a corner of a grid square. This examples shows +Show me all imagery that has low cloud cover (less than 10), and high data coverage (50), as I'd like a cloud free image that is not just a tiny sliver of data. #### Example 3: AND cql2-text (GET) ```http -filter=sentinel:data_coverage > 50 AND eo:cloud_cover < 10 +filter=sentinel:data_coverage > 50 AND eo:cloud_cover < 10 ``` #### Example 3: AND cql2-json (POST) @@ -674,11 +672,11 @@ filter=sentinel:data_coverage > 50 AND eo:cloud_cover < 10 "filter": { "op": "and", "args": [ - { + { "op": ">", "args": [ { "property": "sentinel:data_coverage" }, 50 ] }, - { + { "op": "<", "args": [ { "property": "eo:cloud_cover" }, 10 ] } @@ -687,7 +685,7 @@ filter=sentinel:data_coverage > 50 AND eo:cloud_cover < 10 } ``` -An 'or' is also supported, matching if either condition is true. Though it's not a sensible query you could get images that have full data +An 'or' is also supported, matching if either condition is true. Though it's not a sensible query you could get images that have full data coverage or low cloud cover. ### Example 4: Disjunction with OR @@ -697,7 +695,7 @@ This uses the same queryables as Example 4. #### Example 4: OR cql2-text (GET) ```http -filter=sentinel:data_coverage > 50 OR eo:cloud_cover < 10 +filter=sentinel:data_coverage > 50 OR eo:cloud_cover < 10 ``` #### Example 4: OR cql2-json (POST) @@ -706,14 +704,14 @@ filter=sentinel:data_coverage > 50 OR eo:cloud_cover < 10 { "filter-lang": "cql2-json", "filter": { - "op": "or", + "op": "or", "args": [ - { - "op": ">", + { + "op": ">", "args": [ { "property": "sentinel:data_coverage" }, 50 ] }, - { - "op": "<", + { + "op": "<", "args": [ { "property": "eo:cloud_cover" }, 10 ] } ] @@ -726,7 +724,7 @@ filter=sentinel:data_coverage > 50 OR eo:cloud_cover < 10 The Property-Property Comparisons conformance class permits queryable properties to be used on either side of an operator. This is a generic example, as there are few STAC properties that are comparable in a meaningful way. This example uses a contrived example of two proprietary properties, -`prop1` and `prop2` that are of the same type. +`prop1` and `prop2` that are of the same type. This queryables JSON Schema is used in these examples: @@ -744,7 +742,7 @@ This queryables JSON Schema is used in these examples: }, "prop2" : { "description" : "Property 2", - "type": "integer" + "type": "integer" } } } @@ -759,10 +757,10 @@ filter=prop1 = prop2 #### Example 5: POST with cql2-json ```json -{ +{ "filter-lang": "cql2-json", "filter": { - "op": "eq", + "op": "=", "args": [ { "property": "prop1" }, { "property": "prop2" } @@ -775,7 +773,7 @@ filter=prop1 = prop2 This uses the same queryables as Example 4. -The only temporal operator required is `ANYINTERACTS`. This is effectively that the datetime or interval operands +The only temporal operator required is `ANYINTERACTS`. This is effectively that the datetime or interval operands have any overlap between them. #### Example 6: T_INTERSECTS cql2-text (GET) @@ -790,7 +788,7 @@ filter=datetime T_INTERSECTS INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00: { "filter-lang": "cql2-json", "filter": { - "op": "t_intersects", + "op": "t_intersects", "args": [ { "property": "datetime" }, { "interval" : [ "2020-11-11T00:00:00Z", "2020-11-12T00:00:00Z"] } @@ -801,7 +799,7 @@ filter=datetime T_INTERSECTS INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00: ### Example 7: Spatial Intersection -The only spatial operator that must be implemented for Basic Spatial Operators +The only spatial operator that must be implemented for Basic Spatial Operators is `S_INTERSECTS`. This has the same semantics as provided by the Item Search `intersects` parameter. The `cql2-text` format uses WKT geometries and the `cql2-json` format uses GeoJSON geometries. @@ -818,7 +816,7 @@ filter=S_INTERSECTS(geometry,POLYGON((-77.0824 38.7886,-77.0189 38.7886,-77.0189 { "filter-lang": "cql2-json", "filter": { - "op": "s_intersects", + "op": "s_intersects", "args": [ { "property": "geometry" } , { @@ -852,11 +850,11 @@ filter=S_INTERSECTS(geometry,POLYGON((-77.0824 38.7886,-77.0189 38.7886,-77.0189 ```json { "filter-lang": "cql2-json", - "filter": { - "op": "or" , + "filter": { + "op": "or" , "args": [ - { - "op": "s_intersects", + { + "op": "s_intersects", "args": [ { "property": "geometry" } , { @@ -869,8 +867,8 @@ filter=S_INTERSECTS(geometry,POLYGON((-77.0824 38.7886,-77.0189 38.7886,-77.0189 } ] }, - { - "op": "s_intersects", + { + "op": "s_intersects", "args": [ { "property": "geometry" } , { @@ -884,7 +882,7 @@ filter=S_INTERSECTS(geometry,POLYGON((-77.0824 38.7886,-77.0189 38.7886,-77.0189 ] } ] - } + } } ``` @@ -895,7 +893,7 @@ different sets of properties. For example, a collection of Sentinel 2 data may h `sentinel:data_coverage` and a collection of Landsat 8 data may have a corresponding property `landsat:coverage_percent`, both representing what percentage of a given gridded scene actually contains data. However, we many also want to also include in our result items that do not have a value defined for -either of those properties. +either of those properties. #### Example 9: cql2-text (GET) @@ -909,25 +907,25 @@ filter=sentinel:data_coverage > 50 OR landsat:coverage_percent < 10 OR (sentinel { "filter-lang": "cql2-json", "filter": { - "op": "or", + "op": "or", "args": [ - { - "op": ">=", + { + "op": ">", "args": [ { "property": "sentinel:data_coverage" }, 50 ] }, - { - "op": ">=", - "args": [ { "property": "landsat:coverage_percent" }, 50 ] + { + "op": "<", + "args": [ { "property": "landsat:coverage_percent" }, 10 ] }, { - "op": "and", + "op": "and", "args": [ - { + { "op": "isNull", "args": { "property": "sentinel:data_coverage" } }, { - "op": "isNull", + "op": "isNull", "args": { "property": "landsat:coverage_percent" } } ] @@ -953,16 +951,16 @@ filter=eo:cloud_cover BETWEEN 0 AND 50 { "filter-lang": "cql2-json", "filter": { - "op": "between", - "args": [ - { "property": "eo:cloud_cover" }, - [ 0, 50 ] + "op": "between", + "args": [ + { "property": "eo:cloud_cover" }, + [ 0, 50 ] ] } } ``` -### Example 11: Using LIKE +### Example 11: Using LIKE The LIKE operator allows for pattern-based string matching. @@ -978,9 +976,9 @@ filter=mission LIKE 'sentinel%' { "filter-lang": "cql2-json", "filter": { - "op": "like", - "args": [ - { "property": "mission" }, + "op": "like", + "args": [ + { "property": "mission" }, "sentinel%" ] } @@ -994,11 +992,11 @@ The predefined function `CASEI` allows for case-insensitive comparisons. #### Example 12: cql2-text (GET) ```http -filter=CASEI(provider) == 'coolsat' +filter=CASEI(provider) = 'coolsat' ``` ```http -filter=CASEI(provider) == 'NASA' +filter=CASEI(provider) = 'NASA' ``` #### Example 12: cql2-json (POST) @@ -1007,9 +1005,9 @@ filter=CASEI(provider) == 'NASA' { "filter-lang": "cql2-json", "filter": { - "op": "eq", + "op": "=", "args": [ - { + { "lower" : { "property": "provider" } }, "coolsat" @@ -1022,9 +1020,9 @@ filter=CASEI(provider) == 'NASA' { "filter-lang": "cql2-json", "filter": { - "op": "eq", + "op": "=", "args": [ - { + { "upper": { "property": "provider" } }, "NASA" From 4007eebb313924babc7e6310250ff2364583567e Mon Sep 17 00:00:00 2001 From: David Bitner Date: Thu, 13 Jan 2022 15:25:57 -0600 Subject: [PATCH 2/2] update changelog --- CHANGELOG.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 98e44692..57ca1710 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,7 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Updated all "CQL" usages to "CQL2" - Most conformance class URIs are now prefixed with `http://www.opengis.net/spec/cql2/` instead of `http://www.opengis.net/spec/ogcapi-features-3/` - - Conformance classes `http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/basic-cql`, + - Conformance classes `http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/basic-cql`, `http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/cql-text`, and `http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/cql-json` have had `cql` replaced with `cql2` (in addition to the prefix change) to @@ -35,11 +35,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Temporal operator `ANYINTERACTS` is now `T_INTERSECTS` - Updated Example 3 (now Example 5) to make it clear that property to property comparisons require the Property-Property Comparisons conformance class - - The CQL2 Case-insensitive Comparison + - The CQL2 Case-insensitive Comparison (`http://www.opengis.net/spec/cql2/1.0/conf/case-insensitive-comparison`) conformance class that adds UPPER/LOWER terms or function CASEI for case-insensitive comparison has not been added to this spec yet, since the definition in CQL2 is in flux. -- `service-desc` endpoint may return any service description format, typically a +- `service-desc` endpoint may return any service description format, typically a machine-consumable one (previous restricted required to be OpenAPI 3.0 JSON) - `service-doc` endpoint may return any service description format, typically a human-consumable one (previous restricted required to be HTML) @@ -50,10 +50,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- Collection conformance class URI should be `https://api.stacspec.org/v1.0.0-beta.XXX/collections` instead +- Collection conformance class URI should be `https://api.stacspec.org/v1.0.0-beta.XXX/collections` instead of `http://stacspec.org/spec/api/1.0.0-beta.XXX/extensions/collections` - definition of Item object was missing `properties` as an attribute - Filter Extension - examples of using intervals and timestamps in CQL2 were incorrect and have been fixed +- Filter Extension - examples are updated so that text and json examples are equivalent ## [v1.0.0-beta.4] - 2020-10-05 @@ -80,8 +81,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - "Arithmetic" has been renamed "Arithmetic Expressions" - "Arrays" has been renamed "Array Operators" - "Queryable Second Operand" has been renamed "Property-Property Comparisons" -- The required Link Relations and endpoints for each conformance class now use the wording of 'shall' - instead of 'should'. While this technically changes the semantics, it was generally understood +- The required Link Relations and endpoints for each conformance class now use the wording of 'shall' + instead of 'should'. While this technically changes the semantics, it was generally understood previously the semantics were those of 'shall' (must). ### Deprecated @@ -99,9 +100,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - Query extension not deprecated; recommendation to use Filter (https://github.com/radiantearth/stac-api-spec/pull/157) - Filter Extension conformance classes refactored to better align with STAC API use cases. -- Renamed conformance class "Queryable First Operand" - (https://api.stacspec.org/v1.0.0-beta.3/item-search#filter:queryable-first-operand) to - "Queryable Second Operand" +- Renamed conformance class "Queryable First Operand" + (https://api.stacspec.org/v1.0.0-beta.3/item-search#filter:queryable-first-operand) to + "Queryable Second Operand" (https://api.stacspec.org/v1.0.0-beta.3/item-search#filter:queryable-second-operand) ### Deprecated @@ -110,7 +111,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Remove stac_version and stac_extensions fields in ItemCollection ### Fixed - + ## [v1.0.0-beta.2] - 2020-06-01 ### Added