From 3ef5813d1d2ff0edde50dae3904c9a834a005852 Mon Sep 17 00:00:00 2001 From: "Masih H. Derkani" Date: Thu, 26 Jan 2023 15:41:02 +0000 Subject: [PATCH 1/2] Introduce specification for cascading lookup query parameter It is increasingly useful for the users to be able to search alternative routing systems through the familiar IPNI endpoints. This greatly simplifies the barrier to simply find providers for a given CID, and avoid the need to learn about explicit routing systems such as the DHT. Further, the results can be cached and reused by others considering the fact that a typical IPNI instance will heavily utilise caching mechanisms regardless of cascading lookups. The cascading lookup must be configurable to avoid doubling load on other routing systems in a case where, having searched other routing systems already, the user exclusively wants to search IPNI. For example, Kubo `v0.18.0` is configured by default to lookup records on the DHT and on IPNI. Therefore, automatic cascading of lookups onto the DHT by IPNI would result in duplication of load and waisted effort since Kubo independently looks up the DHT. The changes here introduce a new query parameter, called `cascade` for the lookup APIs, where the user can specify a list of comma separated alternative routing systems to which the query is cascaded in addition to IPNI. By default no cascading occurs. This allows Kubo to gracefully continue functioning while enabling other clients to optionally cascade lookups onto the IPFS DHT. --- IPNI.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/IPNI.md b/IPNI.md index 0455c69..bc22132 100644 --- a/IPNI.md +++ b/IPNI.md @@ -469,6 +469,11 @@ and uses the multihash portion of the CID only. * `cid` - _Required_. The default string representation of the Cid. Currently, Base32 is used for CIDv1 as the encoding for the multibase string and Base58 is used for CIDv0. +##### Query Parameters + +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: + * `ipfs-dht`- The IPFS Kademlia DHT. + ##### Response * `application/json` - JSON encoded [`FindResponse`][find-response-schema]. See [JSON Find Response](#json-find-response) example. @@ -488,6 +493,12 @@ Given a multihash as path parameter, returns a list of its content providers. * `multihash` - _Required_. The Base58 string representation of multihash value. +##### Query Parameters + +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: + * `ipfs-dht`- The IPFS Kademlia DHT. + + ##### Response * `application/json` - JSON encoded [`FindResponse`][find-response-schema]. See [JSON Find Response](#json-find-response) example. @@ -520,6 +531,12 @@ Base58 string representation. See [`FindRequest`][find-request-schema] schema. } ``` +##### Query Parameters + +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: + * `ipfs-dht`- The IPFS Kademlia DHT. + + ##### Response * `application-json` - JSON encoded [`FindResponse`][find-response-schema]. See [JSON Batch Find Response](#json-batch-find-response) example. From b3060a00b941b29952fc099a4f13d4f0e7273d9f Mon Sep 17 00:00:00 2001 From: "Masih H. Derkani" Date: Fri, 27 Jan 2023 15:43:32 +0000 Subject: [PATCH 2/2] Add support for listing cascade options Specify protocol to find out what cascade options are available to a client using `OPTIONS` method and a specific HTTP response header. This enables clients to programmatically determine the alternative routing systems to which a lookup may optionally be cascaded. Add extra wording to clarify that there are no constraints on the order by which results are returned at the presence of `cascade` query option. --- IPNI.md | 31 +++++++++++++++++------- schemas/v1/openapi.yaml | 53 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 74 insertions(+), 10 deletions(-) diff --git a/IPNI.md b/IPNI.md index bc22132..0ccebcc 100644 --- a/IPNI.md +++ b/IPNI.md @@ -459,6 +459,22 @@ to fetch the advertisement chain. An indexer node can be queried over HTTP for a multihash or a CID. This section provides a summary of the HTTP query APIs. A full OpenAPI specification of the APIs can be found [here](schemas/v1/openapi.yaml). + +#### Cascading Lookup + +The HTTP query API supports cascading queries for a given multihash or CID onto alternative routing systems in addition to searching IPNI records. +A client may optionally specify a query parameter with key set to `cascade`, and value set to comma separated alternative routing systems, which are also searched for records. + +The specification imposes no constraints on the order by which the results are returned. +Implementers are free to return results as they are found. + +The alternative routing systems currently supported is: + * `ipfs-dht`: equivalent to searching records on the IPFS network. + +A client may discover the list of alternative routing systems supported by a server via sending `OPTIONS` request. +In response, the server may include `X-IPNI-Allow-Cascade` header key, with value as the comma separated list of alternative routing systems supported. +The absence of this header key implies that the server does not offer cascading lookups. + #### `GET /cid/{cid}` Given a CID as path parameter, returns a list of its content providers. The lookup ignores CID codec @@ -471,8 +487,7 @@ and uses the multihash portion of the CID only. ##### Query Parameters -* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: - * `ipfs-dht`- The IPFS Kademlia DHT. +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. See [Cascading Lookup](#cascading-lookup) ##### Response @@ -495,8 +510,7 @@ Given a multihash as path parameter, returns a list of its content providers. ##### Query Parameters -* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: - * `ipfs-dht`- The IPFS Kademlia DHT. +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. See [Cascading Lookup](#cascading-lookup) ##### Response @@ -533,8 +547,7 @@ Base58 string representation. See [`FindRequest`][find-request-schema] schema. ##### Query Parameters -* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. Supported values: - * `ipfs-dht`- The IPFS Kademlia DHT. +* `cascade` - _Optional_. The comma separated alternative routing systems to which the lookup is cascaded in addition to searching IPNI index records. If unspecified, only IPNI index records are searched. See [Cascading Lookup](#cascading-lookup) ##### Response @@ -667,6 +680,6 @@ The following lists the libraries and implementations of IPNI protocol: Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). -[find-response-schema]: schemas/v1/openapi.yaml#L108 -[provider-record-schema]: (schemas/v1/openapi.yaml#L124) -[find-request-schema]: schemas/v1/openapi.yaml#L144 +[find-response-schema]: schemas/v1/openapi.yaml#L120 +[provider-record-schema]: (schemas/v1/openapi.yaml#L136) +[find-request-schema]: schemas/v1/openapi.yaml#L156 diff --git a/schemas/v1/openapi.yaml b/schemas/v1/openapi.yaml index 54f8b03..542c145 100644 --- a/schemas/v1/openapi.yaml +++ b/schemas/v1/openapi.yaml @@ -1,7 +1,6 @@ openapi: 3.0.3 info: title: IPNI HTTP API - summary: IPNI HTTP query API description: The Interplanetary Network Indexer (IPNI) HTTP query API. version: 0.0.1 paths: @@ -13,6 +12,7 @@ paths: in: path description: The string representation of the CID. required: true + - $ref: '#/components/parameters/Cascade' responses: '200': description: At least one provider was found successfully. @@ -39,6 +39,12 @@ paths: description: Failure occurred while processing the request. content: text/plain: { } + /cid: + options: + description: Returns permitted communication options for lookup by CID. + responses: + 204: + $ref: '#/components/responses/FindOptions' /multihash/{multihash}: get: description: Finds provider records for a given multihash @@ -47,6 +53,7 @@ paths: in: path description: The base58 string representation of the multihash. required: true + - $ref: '#/components/parameters/Cascade' responses: '200': description: At least one provider was found successfully. @@ -74,6 +81,11 @@ paths: content: text/plain: { } /multihash: + options: + description: Returns permitted communication options for lookup by multihash. + responses: + 204: + $ref: '#/components/responses/FindOptions' post: description: Batch finds provider records for a given set of multihashes requestBody: @@ -236,6 +248,45 @@ components: } ] } + parameters: + Cascade: + name: cascade + in: query + description: | + The comma separated list of routing systems to search for providers in addition to IPNI. + The list of supported routing systems can be discovered by reading `X-IPNI-Allow-Cascade` + response header to `OPTIONS` request. + schema: + type: string + required: false + responses: + FindOptions: + description: Request accepted. + headers: + Access-Control-Allow-Origin: + description: | + The origin permitted to the request. + See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin. + schema: + type: string + Access-Control-Allow-Headers: + description: | + The comma separated list of HTTP headers inspected by the server. + See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers + schema: + type: string + Access-Control-Allow-Methods: + description: | + The comma separated list of permitted HTTP methods. + See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Request-Method + schema: + type: string + X-IPNI-Allow-Cascade: + description: | + The comma separated list of supported routing systems onto which the lookup can be cascaded. + The absence of this header implies that the server does not offer cascading lookups. + schema: + type: string externalDocs: description: IPNI Specification url: https://github.com/ipni/specs/blob/main/IPNI.md \ No newline at end of file