Allow passing function refs to Portal.run() ?

[goodboy]

@parity3 had the same thought as I: why can't Portal.run() accept a direct function reference and then we simply lookup the functions module and name much like we do in ActorNursery.run_in_actor()? This would of course have the implicit assumption that the remote actor has the chosen function's module code loaded for use.

The only question is really how to expose both ways on Portal?

• use a different method, say run_func()?
• allow passing an explicit func or ref kwarg?
• allow the first arg to Portal.run() to be either a module name or a func ref and then in the named case make name a required kwarg?

[parity3]

Probably want to go with a separate method names here? Better to be explicit. Another option is for this to be the default, and have a run_serialized_func() method, which sends the source over the wire, or however else you safely serialize a function.

Something like:
run_func(module_path, sub_path, *args, **kwargs)

So you could reference/run classmethods or staticmethods in a sub-namespace inside a module. And it's explicit / the caller knows what is going on.

[goodboy]

Quoting @ryanhiebert here from his comment in #105:

I understand your objection, and this is relatively sane approach to what you're trying to do here. My thinking with this suggestion was that I was wanting to consider tractor to be "trio across machines", which is a really, really cool idea. I think that the names are, essentially, how we would expect to be doing things under the hood. It's how Celery works, too. My question though, is if it really is a public interface, or if it's an implementation detail.

RE trio across machines (and processes) is a big 👍. I think I actually agree with

I think that the names are, essentially, how we would expect to be doing things under the hood.

now after thinking about this more. i previously had a hazy version of what an inter-actor comms API looked like for multi-host setups and I think what I was thinking is wrong. More on this later.

If it's a public interface, then we should expect to see users being very careful about compatibility between different versions of code on different machines. They should want to, and be expected to, consider the sending side and the receiving side separately. The calling side might have no understanding of what code might actually run for a particular message. If a unified vision was desired, it really would have to be built on top of tractor.

I'd even go further that the API for calling code and starting tasks must be explicitly different then the current Nusery API. I think there needs to be a big distinction between starting an actor on the current host versus a remote one even though originally I was thinking this would be seemless / the same API. Feel free to debate this but I'm starting to think any kind of remote actor supervisor needs to be implemented as a layer and API on top of the existing local host equivalent. I also think we need to dig more into how other systems address this, particularly erlang.

That might not the mode that tractor is looking to primarily fill, but that is what I was hoping tractor would be. I think that, internally, it's important to have the mode you're talking about, to make the interface explicit. I just also think that, ideally in my view, it should be a lower-level API for advance uses, rather than the Right Way to use tractor for the general case.

I actually think now after thinking about remote (host) actor spawning that a Portal to a local host actor should be kept explicitly different from a Portal to a remote one. It's because of exactly this

we should expect to see users being very careful about compatibility between different versions of code on different machines

that they need to be so differentiated.

I envision that this layer on top of the tractor plumbing would be able to take a function, convert it to some serialized form, send it over the wire to the remote host, de-serialize it, and run it using the tractor plumbing. How exactly the serialization happens, or the deserialization happens, is an implementation detail that I expect many users need not care about, unless they start dipping into advanced uses.

I would almost prefer this type of RPC/script injection is something that tractor explicitly doesn't support in the core. Someone could add an layer to do this or put it in another dependent project but I actually am somewhat in principle against this model. Imho code should be deployed explicitly through some other mechanism/system.

In some cases it could be looking up a function's module name and path, and sending that. In other cases I could even imagine it serializing the function itself to replay on the remote host, using something like cloudpickle (though I can also imagine impassible challenges attempting to actually do that). I can imagine checks on deserialize that ensure that the destination is running the same code as the sender serialized from.

Understood.

If that interface is not the "default" interface of tractor, then that would seem to me to be an indication that, if such a user-friendly layer were desired, it really should be part of a different package that wraps tractor, instead of a part of tractor itself. Does that seem correct to you?

As mentioned the more I think on it Portal.run() for local host actors should take function references.
If you're trying to run code on another host (or outside the permitted process space) there should be an explicit system in place that determines how that's done and I think that should be reflected in the API even if the API is polymorphic with the local host version to some degree. Maybe even going further and making the RPC API indicative of what kind of RPC is taking place is the correct approach?

If tractor is intending to be that, then I would expect that the registration of the functions to names in a registry should be very explicit, but that does not seem to be the case from the readme, although there's definitely room for me to still not be understanding it correctly.

Fair, the rpc_module_paths mention is mostly a footnote in the docs..

Actually, all of this talking is making me feel like I just haven't spent enough time actually using tractor, to get a good feel for where you're trying to position it. I'm looking for that "trio across processes". You may be targeting a more low-level primitive, or a model that doesn't cleanly fit into what I'm looking for. So take all of my talk above for what they're worth: the idle ramblings of someone with his own ideas, that may not really reflect the ideas you wish to solve.

👌

[goodboy]

@ryanhiebert I think you're going to get your wish on this one.

After thinking it through I can't think of a reason to not encourage passing func references to these methods and any inter-host rpc system will need to be built on top of this regardless.

This will land before #104.

[goodboy]

btw @parity3 take a look at #174 to see if it meets your fancy; will likely get merged this aft!