-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement the pooling instance allocator. #2518
Commits on Mar 5, 2021
-
Refactor module instantiation in the runtime.
This commit refactors module instantiation in the runtime to allow for different instance allocation strategy implementations. It adds an `InstanceAllocator` trait with the current implementation put behind the `OnDemandInstanceAllocator` struct. The Wasmtime API has been updated to allow a `Config` to have an instance allocation strategy set which will determine how instances get allocated. This change is in preparation for an alternative *pooling* instance allocator that can reserve all needed host process address space in advance. This commit also makes changes to the `wasmtime_environ` crate to represent compiled modules in a way that reduces copying at instantiation time.
Configuration menu - View commit details
-
Copy full SHA for b58afbf - Browse repository at this point
Copy the full SHA b58afbfView commit details -
Allow instance allocators control over module compilation.
This commit introduces two new methods on `InstanceAllocator`: * `validate_module` - this method is used to validate a module after translation but before compilation. It will be used for the upcoming pooling allocator to ensure a module being compiled adheres to the limits of the allocator. * `adjust_tunables` - this method is used to adjust the `Tunables` given the JIT compiler. The pooling allocator will use this to force all memories to be static during compilation.
Configuration menu - View commit details
-
Copy full SHA for c8871ee - Browse repository at this point
Copy the full SHA c8871eeView commit details -
Refactor runtime
Table
to support static storage.This commit refactors `Table` in the runtime such that it can be created from a pointer to existing table data. The current `Vec` backing of the `Table` is considered to be "dynamic" storage. This will be used for the upcoming pooling allocator where table memory is managed externally to the instance. The `table.copy` implementation was improved to use slice primitives for doing the copying. Fixes bytecodealliance#983.
Configuration menu - View commit details
-
Copy full SHA for f0d93d1 - Browse repository at this point
Copy the full SHA f0d93d1View commit details -
Store memories and tables on Instance as PrimaryMap.
This commit changes how memories and tables are stored in `Instance`. Previously, the memories and tables were stored as a `BoxedSlice`. Storing it this way requires an allocation to change the length of the memories and tables, which is desirable for a pooling instance allocator that is reusing an `Instance` structure for a new instantiation. By storing it instead as `PrimaryMap`, the memories and tables can be resized without any allocations (the capacity of these maps will always be the configured limits of the pooling allocator).
Configuration menu - View commit details
-
Copy full SHA for dd284ac - Browse repository at this point
Copy the full SHA dd284acView commit details -
Change how Instance stores instantiated memories in the runtime.
This commit changes `Instance` such that memories can be stored statically, with just a base pointer, size, maximum, and a callback to make memory accessible. Previously the memories were being stored as boxed trait objects, which would require the pooling allocator to do some unpleasant things to avoid allocations. With this change, the pooling allocator can simply define a memory for the instance without using a trait object.
Configuration menu - View commit details
-
Copy full SHA for 5beb81d - Browse repository at this point
Copy the full SHA 5beb81dView commit details -
Ensure default allocator is used for instance deallocation.
Handles created with `create_handle` need to be deallocated with the default (on-demand) instance allocator. This commit changes Store such that handles can be added with a flag that is used to force deallocation via the default instance allocator when the Store is dropped.
Configuration menu - View commit details
-
Copy full SHA for 8457261 - Browse repository at this point
Copy the full SHA 8457261View commit details -
Only treat a memory as static when the minimum is also within bounds.
With the change to artificially limit unbounded memories based on Tunables, it's possible to hit the assert where the minimum might exceed the static memory bound. This commit removes the assert in favor of a check to see if the minimum also fits within the static memory bound. It also corrects the maximum bounding to ensure the minimum between the memory's maximum and the configured maximum is used. If it does not fit, the memory will be treated as dynamic. In the case of the pooling instance allocator, the bounds will be checked again during translation and an appropriate error will be returned as dynamic memories are not supported for that allocator.
Configuration menu - View commit details
-
Copy full SHA for 3bb145f - Browse repository at this point
Copy the full SHA 3bb145fView commit details -
Implement allocating fiber stacks for an instance allocator.
This commit implements allocating fiber stacks in an instance allocator. The on-demand instance allocator doesn't support custom stacks, so the implementation will use the allocation from `wasmtime-fiber` for the fiber stacks. In the future, the pooling instance allocator will return custom stacks to use on Linux and macOS. On Windows, the native fiber implementation will always be used.
Configuration menu - View commit details
-
Copy full SHA for 16ca5e1 - Browse repository at this point
Copy the full SHA 16ca5e1View commit details -
Implement the pooling instance allocator.
This commit implements the pooling instance allocator. The allocation strategy can be set with `Config::with_allocation_strategy`. The pooling strategy uses the pooling instance allocator to preallocate a contiguous region of memory for instantiating modules that adhere to various limits. The intention of the pooling instance allocator is to reserve as much of the host address space needed for instantiating modules ahead of time and to reuse committed memory pages wherever possible.
Configuration menu - View commit details
-
Copy full SHA for e71ccbf - Browse repository at this point
Copy the full SHA e71ccbfView commit details -
Implement user fault handling with
userfaultfd
on Linux.This commit implements the `uffd` feature which turns on support for utilizing the `userfaultfd` system call on Linux for the pooling instance allocator. By handling page faults in userland, we are able to detect guard page accesses without having to constantly change memory page protections. This should help reduce the number of syscalls as well as kernel lock contentions when many threads are allocating and deallocating instances. Additionally, the user fault handler can lazy initialize linear memories of an instance (implementation to come).
Configuration menu - View commit details
-
Copy full SHA for a2c4391 - Browse repository at this point
Copy the full SHA a2c4391View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5b2f878 - Browse repository at this point
Copy the full SHA 5b2f878View commit details -
Skip the stack tests on Windows.
As Windows uses the native fiber implementation, the stack tests should be ignored on Windows as the implementation intentionally errors when handing out stacks.
Configuration menu - View commit details
-
Copy full SHA for a82f1a3 - Browse repository at this point
Copy the full SHA a82f1a3View commit details -
Implement on-demand memory initialization for the uffd feature.
This commit implements copying paged initialization data upon a fault of a linear memory page. If the initialization data is "paged", then the appropriate pages are copied into the Wasm page (or zeroed if the page is not present in the initialization data). If the initialization data is not "paged", the Wasm page is zeroed so that module instantiation can initialize the pages.
Configuration menu - View commit details
-
Copy full SHA for f5c4d87 - Browse repository at this point
Copy the full SHA f5c4d87View commit details -
Refactor
initialize_vmcontext
.This was originally written to support sourcing the table and memory definitions differently for the pooling allocator. However, both allocators do the exact same thing, so the closure arguments are no longer necessary. Additionally, this cleans up the code a bit to pass in the allocation request rather than having individual parameters.
Configuration menu - View commit details
-
Copy full SHA for 9091f13 - Browse repository at this point
Copy the full SHA 9091f13View commit details -
Configuration menu - View commit details
-
Copy full SHA for f170d0b - Browse repository at this point
Copy the full SHA f170d0bView commit details -
Use slice::fill for filling tables.
Now that `slice::fill` is stable, update the table implementation in the runtime to use it.
Configuration menu - View commit details
-
Copy full SHA for f48d1e2 - Browse repository at this point
Copy the full SHA f48d1e2View commit details -
Update the rustc badge to better reflect the supported version.
Wasmtime documentation says stable is the supported rustc version, and that's what we test with CI, so the badge should reflect that. Wasmtime doesn't even build with 1.37 any longer anyway.
Configuration menu - View commit details
-
Copy full SHA for f533df0 - Browse repository at this point
Copy the full SHA f533df0View commit details -
Switch CI back to latest nightly.
The issue that required the pin to the older version has been resolved. This keeps the x64 backend tests on an older nightly version to support the `-Z` flags being passed without having to update Cargo.toml to a new feature resolver version. The doc task is also kept on the older nightly for the same reason.
Configuration menu - View commit details
-
Copy full SHA for 89d3b5d - Browse repository at this point
Copy the full SHA 89d3b5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for a481e11 - Browse repository at this point
Copy the full SHA a481e11View commit details -
Last minute code clean up to fix some comments and rename `address_space_size` to `memory_reservation_size` to better describe what the option is doing.
Configuration menu - View commit details
-
Copy full SHA for 505437e - Browse repository at this point
Copy the full SHA 505437eView commit details -
Fix a bad merge with the `async` feature that accidentally removed the allocation of fiber stacks via the instance allocator.
Configuration menu - View commit details
-
Copy full SHA for 4e83392 - Browse repository at this point
Copy the full SHA 4e83392View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ee2b87 - Browse repository at this point
Copy the full SHA 5ee2b87View commit details -
* Add `anyhow` dependency to `wasmtime-runtime`. * Revert `get_data` back to `fn`. * Remove `DataInitializer` and box the data in `Module` translation instead. * Improve comments on `MemoryInitialization`. * Remove `MemoryInitialization::OutOfBounds` in favor of proper bulk memory semantics. * Use segmented memory initialization except for when the uffd feature is enabled on Linux. * Validate modules with the allocator after translation. * Updated various functions in the runtime to return `anyhow::Result`. * Use a slice when copying pages instead of `ptr::copy_nonoverlapping`. * Remove unnecessary casts in `OnDemandAllocator::deallocate`. * Better document the `uffd` feature. * Use WebAssembly page-sized pages in the paged initialization. * Remove the stack pool from the uffd handler and simply protect just the guard pages.
Configuration menu - View commit details
-
Copy full SHA for a464465 - Browse repository at this point
Copy the full SHA a464465View commit details -
* More use of `anyhow`. * Change `make_accessible` into `protect_linear_memory` to better demonstrate what it is used for; this will make the uffd implementation make a little more sense. * Remove `create_memory_map` in favor of just creating the `Mmap` instances in the pooling allocator. This also removes the need for `MAP_NORESERVE` in the uffd implementation. * Moar comments. * Remove `BasePointerIterator` in favor of `impl Iterator`. * The uffd implementation now only monitors linear memory pages and will only receive faults on pages that could potentially be accessible and never on a statically known guard page. * Stop allocating memory or table pools if the maximum limit of the memory or table is 0.
Configuration menu - View commit details
-
Copy full SHA for ff840b3 - Browse repository at this point
Copy the full SHA ff840b3View commit details -
* Don't reexport types from `wasmtime_runtime` from the `wasmtime` crate. * Add more comments.
Configuration menu - View commit details
-
Copy full SHA for a4084db - Browse repository at this point
Copy the full SHA a4084dbView commit details -
* Add more overflow checks in table/memory initialization. * Comment for `with_allocation_strategy` to explain ignored `Config` options. * Fix Wasmtime `Table` to not panic for type mismatches in `fill`/`copy`. * Add tests for that fix.
Configuration menu - View commit details
-
Copy full SHA for a719076 - Browse repository at this point
Copy the full SHA a719076View commit details
Commits on Mar 6, 2021
-
Make the storage of
wasmtime_runtime::Table
consistent.This change makes the storage of `Table` more internally consistent. Elements are stored as raw pointers for both static and dynamic table storage. Explicitly storing elements as pointers removes assumptions being made by the pooling allocator in terms of the size and default representation of the elements. However, care must be made to properly clone externrefs for table operations.
Configuration menu - View commit details
-
Copy full SHA for 1a04939 - Browse repository at this point
Copy the full SHA 1a04939View commit details -
Fail module translation for segments with overflowing offset+length.
This commit fails translation of modules that have an segment offset, when added to the data length, overflows.
Configuration menu - View commit details
-
Copy full SHA for 9801c68 - Browse repository at this point
Copy the full SHA 9801c68View commit details -
Run wast tests with both instance allocators.
This commit adds a "pooling" variant to the wast tests that uses the pooling instance allocation strategy. This should help with the test coverage of the pooling instance allocator.
Configuration menu - View commit details
-
Copy full SHA for 57dfe99 - Browse repository at this point
Copy the full SHA 57dfe99View commit details -
Extract out finding a passive segment.
This commit extracts out a common pattern of finding a passive element or data segment into a `find_passive_segment` method.
Configuration menu - View commit details
-
Copy full SHA for 8e51aef - Browse repository at this point
Copy the full SHA 8e51aefView commit details
Commits on Mar 8, 2021
-
* Improve comments. * Drop old table element *after* updating the table. * Extract out the same `cfg_if!` to a single constant.
Configuration menu - View commit details
-
Copy full SHA for 7a93132 - Browse repository at this point
Copy the full SHA 7a93132View commit details -
Move linear memory faulted guard page tracking into
Memory
.This commit moves the tracking for faulted guard pages in a linear memory into `Memory`.
Configuration menu - View commit details
-
Copy full SHA for 5fa0f8d - Browse repository at this point
Copy the full SHA 5fa0f8dView commit details -
Use
anyhow::Error
in instantiation errors.This commit updates the error enums used in instantiation errors to encapsulate an `anyhow::Error` rather than a string.
Configuration menu - View commit details
-
Copy full SHA for 623290d - Browse repository at this point
Copy the full SHA 623290dView commit details