Circular rebalancing

[Kixunil]

It'd be great to have support for circular rebalancing. The easiest API to work with would allow to specify outgoing channel, incoming channel and amount. However it doesn't need to be that advanced to be useful.

A simpler approach would be to implement #1068 and also add a flag to sendtoroute to allow sending a payment to self. The client could then write this simple algorithm:

1. find route from the counterparty node belonging to outgoing channel to self
2. prepend the route with the ID of the counterparty node belonging to outgoing channel
3. use sendtoroute with the resulting route and self-payment enabled

This was also discussed in Ride-The-Lightning/RTL#535

[t-bast]

This is the kind of things we want a plugin to implement, this shouldn't have to be done in eclair-core.
I believe plugins have everything they need to implement it, we just need someone to spend time writing it!

[Kixunil]

Is there some kind of extended API that allows for implementing it? I could look into it but I really, really don't want to re-implement path finding. Also, just realized the algorithm above would not work without blacklisting outgoing channel. It could also be done the other way around if findroute supported blacklisting of channels.
Also I'm not convinced self-sending is enabled. How could a plugin enable it?

BTW do we mean the same thing by "plugin"? I was thinking about something that uses the existing REST API, but maybe there's a different way to do it?

[t-bast]

Also I'm not convinced self-sending is enabled.

That's right, but if someone was serious about implementing a plugin that does this, we could remove this requirement.

BTW do we mean the same thing by "plugin"? I was thinking about something that uses the existing REST API, but maybe there's a different way to do it?

The REST API is by design limited in what it can do, I don't mean that when I say plugin.
A plugin integrates much more deeply inside the node: https://github.com/acinq/eclair#Plugins
@btcontract has examples of advanced eclair plugins (probably on his own repository).

[akumaigorodski]

yes, these two:
https://github.com/btcontract/plugin-hosted-channels
https://github.com/btcontract/plugin-chainswap

both are work in progress, I think once I'm done with them I'll make a manual about what a plugin can do and how one should start with them.

[Kixunil]

That's right, but if someone was serious about implementing a plugin that does this, we could remove this requirement.

I'm somewhat seriously interested. Unfortunately, I have literally zero experience with Scala, so firstly I will have to learn it a bit and then also figure out how ACINQ and its plugins work, which from cursory look is not very obvious. On top of that, I'm insanely busy, so it'll take some time until I can even start. If anyone is willing to help, it'd be highly appreciated.

In case of enabling self-payments, it'd still be a good thing to add an explicit flag for opt-in to avoid problems.

Thank you both for all the info so far!

[t-bast]

We do need more documentation and guides around plugins, we planned to do that once people started writing some (to ensure the guides and documentation correctly solves people's pain points). Once we have a set of known community plugins, it's much simpler for new ones to be written: just look at existing plugins and adapt.

Note that plugins can theoretically be written in any JVM-compatible language (Java is an option, even though I much prefer Scala).

In case of enabling self-payments, it'd still be a good thing to add an explicit flag for opt-in to avoid problems.

That's the kind of thing that the plugin could declare: we can probably authorize this only for specific plugin route requests, which will ensure it doesn't bleed into normal payment scenarios.

[Kixunil]

Right, I thing this kind of plugin is very useful and should be among those first plugins. But if someone skilled is interested in writing other plugins first, that's fine.

I guess I could try with Java but I know all the reasons to prefer Scala and I theoretically prefer it as well. :) Syntax is pretty much the only thing I remember from Java anyway - didn't use it for years.

ensure it doesn't bleed into normal payment scenarios.

I actually think it could be useful to allow self-payments in some other cases. Say for example there's a plugin (or RPC proxy) that allows managing virtual accounts and giving access to them to external services. Then maybe it'd be good to have transfer between these accounts also registered in normal Eclair database to maintain synchronized history. I'm not entirely sure how much is this useful.

[t-bast]

Do you think this feature will still be useful once splicing is available?
Off-chain circular rebalancing simply cannot work reliably since it's a closed system.

[Kixunil]

Unsure how much useful but it could.

Consider three nodes A, B, C having three channels between them A - B, B - C, C - A. If it happens that A sends a lot of sats to B, B sends a lot of sats to C, C sends a lot of sats to A, (assuming independent transactions) so that channels are not balanced, then rebalancing in the opposite direction can improve speed of payments. Without it a node trying to route between e.g. A and B would have to try both possibilities with 50% chance of succeeding at first attempt. Each retry takes some time so less retries - better UX.

I can't say how much the market values speed and reliability but given various complains from people ("too slow for paying in grocery store") it's likely somewhat important.

[t-bast]

True, it could be useful in that hypothetical scenario.
Let's keep this issue open then.

[t-bast]

Done with #1969

[ecdsa]

@Kixunil here is a (very crude) bash script for eclair rebalancing: https://github.com/spesmilo/eclair-tools/blob/master/rebalance