Ideas for things we can work on 💡

[ra0x3]

What is this?

• We contribute to projects according to "Milestones"
• Whilst contributing to those milestones, we sometimes have down time, or time that our top priority is blocked (for whatever reason)
• It could be nice to have a list of feature ideas that we could work on if we're not actively contributing to a milestone
• Further, it could even be the case where some of these ideas are considered during the creation of milestones

Is this list final?

• Absolutely not
• If you have any indexer-related ideas, feel free to add a feature/idea to this list using the same format of the ideas that have already been listed in the Ideas section.
• Let's try to make sure that all ideas are actionable/workable - meaning they can be completed in some reasonable amount of time (e.g., 3 weeks)
  • You don't have to present a fully flushed out issue - as you can see, a number of the ideas below are just a few bullet points (with no accompanying issue)
    • If the feature/idea gets picked up - we can create a more specific/flushed-out issue at that time

Ideas

• More advanced block explorer backend functionality

  • We have our example block explorer backend
  • Which exact features will most likely be a Product decision

• Pull remote contract ABI #749

  • Case: If I want to download contract ABI to build an indexer for that contract, but I don't have the contract locally
  • Just downloading a JSON blob into the contracts/ folder of an index project
    • ABI bytes could be stored as Blob
  • https://thegraph.com/docs/en/deploying/hosted-service/#from-an-existing-contract

• Public vs Private indices

  • All indices are public (as of today)
  • With auth, all indices would be private
  • Indices could signal (via the manifest) that they'd like to be public

• Support authentication

  • feature: authentication mechanism for index operators #240
  • Depends on how https://github.com/FuelLabs/fuel-deployment/issues/106 is received
  • As an index operator you don't want the world uploading assets to your service
  • Authentication would no-doubt be coupled with Fuel wallet and/or SDK wallet functionality

• Native execution

  • feature: native execution #389

• Indexer playground

  • https://github.com/FuelLabs/fuel-deployment/issues/106
    • ⚠️ Indexer project related issues for this work are pending the reception of this ☝️ issue
  • https://thegraph.com/docs/en/deploying/deploying-a-subgraph-to-studio/
  • https://thegraph.com/docs/en/deploying/deploying-a-subgraph-to-studio/#installing-graph-cli

• Indexer constraints

  • Add constraints to indexers so that indexers can't use too many resources
  • This is advantageous to "index operators" (of which Fuel would be one, if we created a playground)
  • You don't want some schmuck writing an index that exhausts your VM resources
  • Dependent on the reception of https://github.com/FuelLabs/fuel-deployment/issues/106
  • Add constraints to indexer services for playground #734

• Indexer migrations

  • feature: index migrations #382

• Create an indexer registry

  • feature: index asset bundle registry #400

• Websockets

  • feature: persistent read-only event subscription functionality #390

• Extend the FFI

  • So far our ORM-ish FFI supports load and save
  • We could potentially expand on that with other methods such as filter in order to retrieve several entities of the same type T matching certain criteria
  • Extend and optimize the ORM functionality of our FFI #860

• NoSQL support

  • An RDS handles 85% of indexing use-cases, but for cases where speed read/write time are most important a NoSQL/cache-based backed can be better
  • InMemory for WASM and Native execution
  • Redis (or something of that nature) for Native execution
    • WASM does not support Redis or RocksDB I don't believe 🤔

• Integrate with fuel-core TX pool

  • Create subscription to fuel-core TX pool #1418
  • We've had several users and people mention the value of the TX pool for MEV
  • Would fetch transactions via fuel-gql-client web API
  • Check with Brandon Kite on specifics (Client team might have to do a small bit of work here)
  • We might have to write an abstraction to "manage" the TX pool so that index users can access these TXs via a simple API

• Investigate tooling for creating indices in TypeScript (probably more of a research project)

  • Investigate usage of Typescript in indexers #1416
  • Users are currently required to know and write Rust to create an index; it would be great if we could allow them to use TypeScript
  • TypeScript has WASM support through AssemblyScript, but it may not be fully-featured enough for this use case
  • We should investigate whether its possible to leverage fuels-ts to allow for indexing upon Fuel types
  • Additionally, we would need to find a way to parse user-generated indexer schema files; they're constructed using valid GraphQL syntax so if we could find a way to generate type defs on the fly for a user's schema, that would be great
  • fuels-ts already does something similar when generating types from contract ABIs

• Consider adding advanced filtering capabilities to GraphQL API

  • Currently, the API only supports an "equals" filter, i.e. WHERE X = Y.
  • The GraphQL specification is intentionally vauge as to how filtering should be supported.
  • We should decide whether we want to allow for more advanced filtering and how to do so(especially if we're going to be maintaining the GrapQL API from fuel-core)
  • Examples of searching and filtering can be found here: https://dgraph.io/docs/graphql/queries/search-filtering/#query-that-filters-objects-by-predicate
  • Support GraphQL pagination #656
  • Add filtering to GraphQL API queries #637
  • Integrate web server, GraphQL layer, and data layer #1145

• Create a GUI for simple interface

  • Currently the primary way a user can interact with the indexer is via the CLI. Much of the functionality remains hidden to the user unless the really study the --help or docs. It would be great to have a simple interface where users could run the various plugin commands, with check boxes and input fields to pass values as args or set flags.
  • Think about the AWS console interface for example, simple UI, powerful features. For many AWS users, the UI is the primary way to use the tech.
  • The GUI could be browser based or desktop based.

• Create contract callpath(s) via contract ABI

  • We can get the selector IDs and function names from the Sway JSON ABI
  • We can use these selector IDs and function names to create a "callpath" -- showing the trace/route of a given contract's calls

[0xmovses]

• create a bash script inside /scripts for complete local dev set up
  • The script would spin up a fuel node, the indexer and / or the indexer-api (depending on flags).
  • It would output some nice ASCII art and some message for flags options.
  • This would give developers a pleasant experience and reduces the amount of commands needed for a full testing setup. (DB would have to be created before this step).

[ra0x3]

@rvmelkonian

• I think if we have the time we could use your idea ☝🏽 as an opportunity to get super creative
• Like, dope, colorful ASCII art
• Maybe something even interactive (like a step-through tutorial kinda thing)
• That would not only be interesting for devs (your original use-case I believe) but more so for users (our main focus)

[ra0x3]

@deekerno @rvmelkonian I would be super interested in seeing if we can get "Investigate tooling for creating indices in TypeScript (probably more of a research project)" on the next milestone

• That wouldn't mean "it's totally possible we're 100% gonna do it"
• More so it would mean, let's see if we can create a PoC and see what we're missing with regard to making something production ready
• On a literal level, we would just create a PoC by writing the TS by hand
• From there we would try to see what functionality can be abstracted away
• We already have TS ABIGen, so we would just need to create TS-GraphQL-schema-gen (i.e., GraphQL schema -> TS)
• Acceptance (whether PoC or full on production-ready implementation) can be a hello-world-ts example

Thoughts?

[deekerno]

Thoughts?

I'm down to try it out as it's been percolating in the back of my mind for a few months now. My biggest concern is how much of a difference there is between AssemblyScript and TypeScript since they're technically not the same and I'm not sure what subset of TS is contained in AS. Nevertheless, I'm down to see what comes out of it.

[0xmovses]

Thoughts?

I think this would be really useful tooling as most eth web3 devs are used to used to using Subgraphs for querying their contracts, which is all TS. I don't think we would need AssemblyScript though as we could should do GraphQL schema -> TS -> RustIngest -> Wasm?

What would be the flow here? Could a user write out their whole indexer in TS?

[deekerno]

I think this would be really useful tooling as most eth web3 devs are used to used to using Subgraphs for querying their contracts, which is all TS. I don't think we would need AssemblyScript though as we could should do GraphQL schema -> TS -> RustIngest -> Wasm?

What's RustIngest? Never heard of it and I can't seem to find anything about it.

What would be the flow here? Could a user write out their whole indexer in TS?

When I originally pitched it, my thought was that a user that knows TS but not Rust could write their index, compile it to WASM, and deploy it to the indexer service as they would with any Rust-based WASM module.

[0xmovses]

RustIngest is a word I just made up, sorry for the time you spent googling this! It was just a placeholder for: some code that could "Ingest" the TS code, convert it to rust, execute in Rust, creating the WASM binary. If we don't have this Rust conversion and execution, then yes I think we would need to use AssemblyScript, but that seems like a major unknown.