Is there any plan for a "header-crate"

[LouChiSoft]

Hi, I come from a C++ background where header/library combos are the standard and I have been looking at what the Rust ecosystem has to provide that functionality. I've come across abi_stable and now stabby and I was wondering if you guys have any plans for a (preferable automated) header-crate that manages all the let stable_fn = lib.get_stabbied::<extern "C" fn(u8)>(b"stable_fn").unwrap(); style boilerplate code that seems to be required when using stabby.

While I am sure not trivial, it would be very convenient if you could mark all the code you want exposed in your library and then building it returns a .dll/.so and a seperate header-crate for use in other projects. Mainly asking because I want to implement a plugin system for a small Rust renderer I am writing and a header-crate/library combo seems like the most ideal way to share said plugins.

[p-avital]

Hi there,

Splitting "headers" and "library" in Rust is a bit more complicated, considering that's not how the language is designed at all, but I'm not certain that's actually what you're after.

To make sure there's no misunderstanding, there are multiple ways to dynamically link, here are equivalences between Rust and C++:

• Runtime linkage: this is your good old dlopen+dlsym(lib, "symbol") combo, equivalent to libloading::Library::new and lib.import(b"symbol"). The turbofish in your example is equivalent to the pointer cast you'd have in dlsym: it's necessary for the compiler to know what you expect that symbol's type to be.
• Load time linkage: this is what you use when you #include <library_headers> and tell your compiler to link that library dynamically. The way you do that in Rust is typically to write a library-sys crate where you'd define an extern block and tell the compiler how to link that extern block; you'd then add that -sys crate to your dependencies.

You seem to be interested in the former (makes sense for a plugin system), so here's my suggestion. But first: do consider to go the way of the IPC plugin system rather than dynamically linked plugin system; IPC plugins have many advantages, at the cost of serialization and IPC communication overhead. Since you're talking about a renderer, the performance hit might be an issue, so here's my suggestion for the DL plugin route:

• Make a <project>-plugins-core crate which defines the API you expect from your plugins as a trait.
• Make your plugins import that crate, and define some

use stabby::alloc::boxed::Box;
use project_plugins_core::PluginTrait;
#[stabby::export]
extern "C" plugin_init(/*args*/) -> stabby::dynptr!(Box<dyn PluginTrait>)

• Profit! You can now use import_stabbied::<extern "C" fn(...)->stabby::dynptr!(...)>(b"plugin_init") in your host (which must also depent on <project>-plugin-core), call that function, and get a complete handle to your plugin. The type system's got your back as to what's available API-wise, and stabby will ensure that things are indeed ABI-compatible.

Good luck on your plugin endeavours! :)

[LouChiSoft]

Hi, thanks for the input. You're right that Rust is not designed at all for library production and consumption, it's my personal biggest pain point in an otherwise very good lanaguge.

You're also right that for a plugin system I would primarily be looking at dynamic loading at runtime when the user (namely myself for now) chooses to enable it. But I do also have some ideas what would fit more to a standard load time linkage system, just for sharing pre-compiled binaries where I don't need to have full source code compilation or in the rare event where sharing source code is not ideal (I have been trying to get work to adopt Rust for safety and performance reasons, but the fact we work with confidential IP and Rust is heavily in favour of source code compilation means we can't really do that).

I won't pretend I know how it could be implemented (maybe a build script, but who knows) but it would seem like a bit of a missed opportunity when (as far as I can tell) when you mark something as stabby::export it has all the knowledge about that function/struct/enum etc, and there could be something there that generates an identical signature that internally has all the linkage code. Kind of like how on Windows DLLs also come with a .lib that is statically linked to the executable and contains all the linkage code.

Regardless it sounds like the idea of a header/sys crate is maybe out of scope for the project which is fine. But if you never ask you never know 🤣

Thanks

[p-avital]

I've been giving the "allow #[stabby::export] to function as a #[stabby::import] depending on a feature flag" idea some thought, but haven't gone all the way to implementation yet. I'll give it some more thought in the coming days :)

[LouChiSoft]

Thanks for considering it, I will be interested in seeing what your conclusion will be

[p-avital]

Keeping you posted: I think I'll indeed make export take a new argument to allow it to turn into an import. There are still a few unresolved questions that I need to dogfood that feature in order to answer, mainly:

• What is an appropriate cfg flag to select one or the other?
  • Feature flags are the obvious answer, but the fact that they get resolved at workspace-level rather than crate-level would be a painful dev experience.
  • Same goes for custom --cfg, as they work globally within a workspace. These do have the advantage that they don't need to be carried around from one dependency to the next.
  • Ideally, #[cfg(crate_type = "cdylib")] would be a good solution, but that cfg flag does not currently exist
• Is there a way to not have to write the name argument usually required for an import?

Since I'm dogfooding stabby a lot more these days, I might figure something out, but for now I don't have a good answer to these questions.

[LouChiSoft]

I see what you mean. I personally was mainly thinking about this from an consumers persective where they simply get the header crate and a fully compiled binary. At that point you'd only have to use an import feature, but I see that in cases where you are compiling libraries as part of the workspace itself that would be more than a little troublesome to deal with.

Thanks again for the update, I look forward to any potential solution that could get around this and I will see if I can think of something myself.