Expose some more internals publicly

[jimpo]

I need this functionality for my application which uses the wasmtime-jit crate. I can provide more details if necessary, but I think these are reasonable methods to add to the interface.

The most important commit is the first, which adds compile_module to Context. Without this, I don't see a good way to create CompiledModules at all.

The most controversial change I'd guess is publicly exposing CodeMemory since it's pretty clear that it was deliberately kept private. I'm OK dropping that commit if people feel that copying the file is better for some reason (it's already copied twice within the wasmtime project).

[pepyakin]

Hey @sunfishcode and @yurydelendik , it would be great to hear what do you think about this change!

[sunfishcode]

Sorry for being so slow here! I'm taking a look at this now. At a high level, Wasmtime is moving to wasmtime-api as the primary way for users to interact with wasmtime as a library. It doesn't have everything yet, but that's where we're going, so I'd be interested if you've looked at it and have thoughts about what does and doesn't work.

wasmtime-jit may remain usable as a library API, but I myself don't currently have a clear sense of what its level of abstraction should be. If you have thoughts about this, I'd be interested!

At an initial glance, exporting CodeMemory from wasmtime-jit seems better than copying it around. This may change as we get more infrastructure for trampolines, so that hopefully external users won't have to manage CodeMemory instances themselves, but it might be ok for now. I'll post again when I've looked at this more.

[sunfishcode]

Ok, having looked at this more closely, I think this PR is fine. I am also interested in hearing your thoughts about wasmtime-api. I expect it likely won't have everything you need right now, but it'd be interesting to hear what kinds of things you need that it doesn't have.

[jimpo]

@sunfishcode Thanks!

I just gave wasmtime-api another brief read and the API does seem to be getting better since the last time I looked at it. A few thoughts:

1. We are using the CompiledModule abstraction as it is important for us to be able to reinstantiate a module with as little overhead as possible. Do you think there is another way to achieve this currently or would you be open to adding something like that to wasmtime-api?
2. The API seems to have a lot more layers of indirection. In particular, seemingly everything is wrapped in a HostRef. What is the purpose of that?
3. The Func also seems a bit klunky as it generates a new instance handle for each external function. While we are also using dynamically generated trampolines (code is pretty much copied from wasmtime-api at the moment), all of the external functions are registered on the same instance handle. Maybe that's not that big a deal though... We also were strongly considering generating the external functions with a macro (like syscalls in wasmtime-wasi). It does seem like a nice option to have available to reduce overhead in making native calls. Thoughts on whether that's useful or an unnecessary optimization?

[yurydelendik]

We are using the CompiledModule abstraction as it is important for us to be able to reinstantiate a module with as little overhead as possible

The wasm-c-api has concept of wasm_module_serialize / wasm_module_deserialize. It is in TODO list for wasmtime-api

The API seems to have a lot more layers of indirection. In particular, seemingly everything is wrapped in a HostRef.

Embedders can attach custom data to any wasmtime-api object (including Instance, Func and AnyRef), and track their destruction. The HostRef provides the above functionality.

The Func also seems a bit klunky as it generates a new instance handle for each external function... We also were strongly considering generating the external functions with a macro (like syscalls in wasmtime-wasi).

It is work-in-progress. But what you see now is a worst/generic case. There is #364 to avoid any instance and to reduce trampoline generation.

[pepyakin]

Would be great to have a new version released on crates.io!