Implement the pooling instance allocator.

[peterhuene]

This PR implements the pooling instance allocator.

The allocation strategy can be set with Config::with_allocation_strategy.

The pooling strategy uses the pooling instance allocator to preallocate a
contiguous region of memory for instantiating modules that adhere to various
limits.

The intention of the pooling instance allocator is to reserve as much of the
host address space needed for instantiating modules ahead of time.

This PR also implements the uffd feature in Wasmtime that enables
handling page faults in user space; this can help to reduce kernel lock
contention and thus increase throughput when many threads are
continually allocating and deallocating instances.

See the related RFC.

[peterhuene]

Hmm, MADV_DONTNEED may not be implemented for aarch64-linux. I'll look into the test failure soon.

[github-actions]

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "cranelift", "cranelift:wasm", "wasmtime:api"

Thus the following users have been cc'd because of the following labels:

• peterhuene: wasmtime:api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[cfallin]

Hmm, MADV_DONTNEED may not be implemented for aarch64-linux. I'll look into the test failure soon.

The aarch64 tests run on qemu, which has this lovely bit of implementation in its syscall handling (link):

    case TARGET_NR_madvise:
        /* A straight passthrough may not be safe because qemu sometimes
           turns private file-backed mappings into anonymous mappings.
           This will break MADV_DONTNEED.
           This is a hint, so ignoring and returning success is ok.  */
        return 0;

At some point we'd ideally run our tests on real hardware, but for now I suspect the best option would just be to disable the unit test and anything that depends on the zeroing semantics. (Or maybe a feature flag to eplicitly memset/bzero instead? That's bad if it was only sparsely committed though...)

[alexcrichton]

We already have a different flag for running in qemu to reduce virtual memory usage overhead since QEMU behaves differently than native processes in that respect, so perhaps that could also be where we configure "hey wasmtime madvise doesn't work as expected"

[github-actions]

Subscribe to Label Action

cc @peterhuene

This issue or pull request has been labeled: "wasmtime:c-api"

Thus the following users have been cc'd because of the following labels:

• peterhuene: wasmtime:c-api

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

[peterhuene]

@alexcrichton I think all of the feedback has now been addressed. FYI: the commit you stopped your review at was "fix bad merge".

This is also running the wast tests with the pooling allocator (+uffd on linux).

My TODO list following this PR:

• Investigate fuzzing the pooling allocator
• Move the fiber creation into the allocators (Wasmtime: lift fiber creation implementation into instance allocator #2708)

[alexcrichton]

Everything here's looking great to me, thanks again for being patient with me :)

As one final follow-up question as well, in the future (not as part of this PR since it's fine as-is), do you think it would be possible to use malloc/free to manage instance/table pools? The memory pool makes sense to use mmap directly since we're doing such funky things with permissions (and it's all page-aligned anyway), and the stack pool makes sense since it's all page aligned too. For instance/tables, however, it seems like there will inevitable be some degree of fragmentation because we round up to page sizes, but I don't think it's strictly necessary (especially now that uffd isn't watching those areas) and we could use malloc/free perhaps?

Otherwise I've got a few more questions/follow-ups below, but otherwise r=me

[peterhuene]

Regarding the page alignment for instances and tables: we don't need it for instances for the pooling allocator, but we do need it for tables because the pooling allocator "decommits" table memory by page and you don't want any page to have elements from multiple tables.

The reason instances are paged-aligned is that this is an artifact from when the pooling allocator had one giant mmap'd region and the memories/tables of the instance came next, so they needed page-alignment back then.

I'll fix the instance pool to not page-align the instances and instead align to Instance's alignment requirements in a follow-up PR.

Regarding malloc/free for instances and tables, I think there's no inherent reason why they can't be used, but part of the intention of this design was to preallocate both (a desirable thing to do for a service) and as Instance is a magical DST, managing one's own memory mapping is probably the easiest way to accomplish that.