IPFS fetches files via graphsync #6831
Labels
Comments
Jorropo
added a commit
that referenced
this issue
Nov 17, 2023
Updates: #9396 Closes: #6831 Closes: #6208 Currently the Graphsync server is not widely used due to lack of compatible software. There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo. There exists some in the filecoin ecosystem but we are not aware of uses with Kubo. Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past. Kubo is consistently one of the fastest software to update to a new go-libp2p release. `go-graphsync` is also developped on many concurrent branches. The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run. It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions. This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature. We are then removing the graphsync server experiment. For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer. If you think this is a mistake and we should put it back you should try to answer theses points: - Find a piece of opensource code which uses a graphsync client to download data from Kubo. - Why is Trustless-Gateway-over-Libp2p not suitable instead ? - Why is bitswap not suitable instead ? Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Jorropo
added a commit
that referenced
this issue
Nov 18, 2023
Updates: #9396 Closes: #6831 Closes: #6208 Currently the Graphsync server is not widely used due to lack of compatible software. There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo. There exists some in the filecoin ecosystem but we are not aware of uses with Kubo. Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past. `go-graphsync` is also developped on many concurrent branches. The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run. It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions. Finaly Kubo is consistently one of the fastest software to update to a new go-libp2p release. This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature. We are then removing the graphsync server experiment. For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer. If you think this is a mistake and we should put it back you should try to answer theses points: - Find a piece of opensource code which uses a graphsync client to download data from Kubo. - Why is Trustless-Gateway-over-Libp2p not suitable instead ? - Why is bitswap not suitable instead ? Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Jorropo
added a commit
that referenced
this issue
Nov 22, 2023
Updates: #9396 Closes: #6831 Closes: #6208 Currently the Graphsync server is not widely used due to lack of compatible software. There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo. There exists some in the filecoin ecosystem but we are not aware of uses with Kubo. Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past. `go-graphsync` is also developped on many concurrent branches. The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run. It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions. Finaly Kubo is consistently one of the fastest software to update to a new go-libp2p release. This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature. We are then removing the graphsync server experiment. For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer. If you think this is a mistake and we should put it back you should try to answer theses points: - Find a piece of opensource code which uses a graphsync client to download data from Kubo. - Why is Trustless-Gateway-over-Libp2p not suitable instead ? - Why is bitswap not suitable instead ? Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Jorropo
added a commit
that referenced
this issue
Nov 22, 2023
Updates: #9396 Closes: #6831 Closes: #6208 Currently the Graphsync server is not widely used due to lack of compatible software. There have been many years yet we are unable to find any production software making use of the graphsync server in Kubo. There exists some in the filecoin ecosystem but we are not aware of uses with Kubo. Even in filecoin graphsync is not the only datatransfer solution available like it could have been in the past. `go-graphsync` is also developped on many concurrent branches. The specification for graphsync are less clear than the trustless gateway one and lack a complete conformance test suite any implementation can run. It is not easily extansible either because selectors are too limited for interesting queries without sideloading ADLs, which for now are hardcoded solutions. Finaly Kubo is consistently one of the fastest software to update to a new go-libp2p release. This means the burden to track go-libp2p changes in go-graphsync falls on us, else Kubo cannot compile even if almost all users do not use this feature. We are then removing the graphsync server experiment. For people who want alternatives we would like you to try the Trustless-Gateway-over-Libp2p experiment instead, the protocol is simpler (request-response-based) and let us reuse both clients and servers with minimal injection in the network layer. If you think this is a mistake and we should put it back you should try to answer theses points: - Find a piece of opensource code which uses a graphsync client to download data from Kubo. - Why is Trustless-Gateway-over-Libp2p not suitable instead ? - Why is bitswap not suitable instead ? Implementation details such as go-graphsync performance vs boxo/gateway is not very interesting to us in this discussion unless they are really huge (in the range of 10x~100x+ more) because the gateway code is under high development and we would be interested in fixing theses.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Go-ipfs must be able to fetch files via graphsync. At the moment, the graphsync interface expects a peer with which to sync and a selector to fetch. It does not support automatically finding providers, splitting requests, etc.
Proposal: Add a (repeated)
--fromflag to instruct IPFS to fetch a file from a specific peer. When passed, go-ipfs should:For now, go-ipfs will not even try to split the request between multiple peers when
--fromis specified and at least one of the specified peers supports graphsync. As we improve graphsync/bitswap, we can add request splitting logic in future PRs.The text was updated successfully, but these errors were encountered: