Skip to content

Commit

Permalink
Limit imported interfaces for composed components (#83)
Browse files Browse the repository at this point in the history
* Limit imported interfaces for composed components

Components that are directly composed will never use any virtualized
interface that it does not already import. Virtualizing additional
interfaces limits the compatibility of the resulting component as it
places an additional requirement on the host environment to satisfy each
import, used or not.

We now inspect the composed component to match the interfaces it imports
with the virtualized capabilities. If the composed component does not
import the respective interface the resulting component will not
virtualize that interface. In effect, use of `--allow-all` will behave
as if the caller specifically approved the interfaces actually used.

The granularity of filtered imports is coarse. Interfaces that support
a consumed capability may still appear as imports even if the composed
component does not consume them explicitly.

Signed-off-by: Scott Andrews <scott@andrews.me>

* Review feedback

- move filtering to be a separate method from finalize
- perform composition in finalize when compose is requested
- update readme

Signed-off-by: Scott Andrews <scott@andrews.me>

---------

Signed-off-by: Scott Andrews <scott@andrews.me>
  • Loading branch information
scothis committed Aug 28, 2024
1 parent 2939de5 commit c451261
Show file tree
Hide file tree
Showing 7 changed files with 238 additions and 63 deletions.
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ wasm-compose = "0.215.0"
wasm-metadata = "0.212.0"
wasm-opt = { version = "0.116.1", optional = true }
wit-component = "0.212.0"
wit-parser = "0.212.0"

[build-dependencies]
anyhow = "1"
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,11 @@ fn main() {
("host.txt", FsEntry::RuntimeFile("/runtime/host/path.txt"))
])));

// compose the virtualization with a component
virt.compose("path/to/component.wasm");
// filter enabled subsystems by imports on the composed component
virt.filter_imports().unwrap();

let virt_component_bytes = virt.finish().unwrap();
fs::write("virt.component.wasm", virt_component_bytes).unwrap();
}
Expand All @@ -195,6 +200,8 @@ By default, when using the `wasi-virt` CLI command, all virtualizations are enab

Selective subsystem virtualization can be performed directly with the WASI Virt library as above (which does not virtualize all subsystems by default). This allows virtualizing just a single subsystem like `env`, where it is possible to virtualize only that subsystem and skip other virtualizations and end up creating a smaller virtualization component.

When directly composing another component, individual subsystems are disabled if the composed component does not import that subsystem. This behavior reduces imports in the output component that are never actually used.

There is an important caveat to this: _as soon as any subsystem uses IO, all subsystems using IO need to be virtualized in order to fully subclass streams and polls in the virtualization layer_. In future this caveat requirement should weaken as these features lower into the core ABI in subsequent WASI versions.

`wasm-tools compose` will error in these scenarios, and better [error messaging](https://github.com/bytecodealliance/wasm-tools/issues/1147) may be provided in future when performing invalid compositions like the above. Missing subsystems can be usually detected from the composition warnings list.
Expand Down
47 changes: 8 additions & 39 deletions src/bin/wasi-virt.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
use anyhow::{bail, Context, Result};
use anyhow::{bail, Result};
use clap::{ArgAction, Parser};
use std::{env, error::Error, fs, path::PathBuf, time::SystemTime};
use std::{error::Error, fs, path::PathBuf};
use wasi_virt::{StdioCfg, WasiVirt};
use wasm_compose::composer::ComponentComposer;

#[derive(Parser, Debug)]
#[command(verbatim_doc_comment, author, version, about, long_about = None)]
Expand Down Expand Up @@ -104,13 +103,6 @@ where
Ok((s[..pos].parse()?, s[pos + 1..].parse()?))
}

fn timestamp() -> u64 {
match SystemTime::now().duration_since(SystemTime::UNIX_EPOCH) {
Ok(n) => n.as_secs(),
Err(_) => panic!(),
}
}

fn main() -> Result<()> {
let args = Args::parse();

Expand Down Expand Up @@ -189,45 +181,22 @@ fn main() -> Result<()> {
fs.allow_host_preopens();
}

if let Some(compose) = args.compose {
virt_opts.compose(compose);
virt_opts.filter_imports()?;
}

let virt_component = virt_opts.finish()?;

let out_path = PathBuf::from(args.out);

let out_bytes = if let Some(compose_path) = args.compose {
let compose_path = PathBuf::from(compose_path);
let dir = env::temp_dir();
let tmp_virt = dir.join(format!("virt{}.wasm", timestamp()));
fs::write(&tmp_virt, virt_component.adapter)?;

let composed_bytes = ComponentComposer::new(
&compose_path,
&wasm_compose::config::Config {
definitions: vec![tmp_virt.clone()],
..Default::default()
},
)
.compose()
.with_context(|| "Unable to compose virtualized adapter into component.\nMake sure virtualizations are enabled and being used.")
.or_else(|e| {
fs::remove_file(&tmp_virt)?;
Err(e)
})?;

fs::remove_file(&tmp_virt)?;

composed_bytes
} else {
virt_component.adapter
};

if virt_component.virtual_files.len() > 0 {
println!("Virtualized files from local filesystem:\n");
for (virtual_path, original_path) in virt_component.virtual_files {
println!(" - {virtual_path} : {original_path}");
}
}

fs::write(&out_path, out_bytes)?;
fs::write(&out_path, virt_component.adapter)?;

Ok(())
}
107 changes: 103 additions & 4 deletions src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
use anyhow::{Context, Result};
use anyhow::{bail, Context, Result};
use serde::Deserialize;
use std::env;
use std::{env, fs, path::PathBuf, time::SystemTime};
use virt_deny::{
deny_clocks_virt, deny_exit_virt, deny_http_virt, deny_random_virt, deny_sockets_virt,
};
use virt_env::{create_env_virt, strip_env_virt};
use virt_io::{create_io_virt, VirtStdio};
use walrus_ops::strip_virt;
use wasm_compose::composer::ComponentComposer;
use wasm_metadata::Producers;
use wit_component::{metadata, ComponentEncoder, DecodedWasm, StringEncoding};
use wit_parser::WorldItem;

mod data;
mod stub_preview1;
Expand Down Expand Up @@ -58,6 +60,8 @@ pub struct WasiVirt {
pub random: Option<bool>,
/// Disable wasm-opt run if desired
pub wasm_opt: Option<bool>,
/// Path to compose component
pub compose: Option<String>,
}

pub struct VirtResult {
Expand Down Expand Up @@ -128,6 +132,67 @@ impl WasiVirt {
self.wasm_opt = Some(opt);
}

pub fn compose(&mut self, compose: String) {
self.compose = Some(compose);
}

/// drop capabilities that are not imported by the composed component
pub fn filter_imports(&mut self) -> Result<()> {
match &self.compose {
Some(compose) => {
let imports = {
let module_bytes = fs::read(compose).map_err(anyhow::Error::new)?;
let (resolve, world_id) = match wit_component::decode(&module_bytes)? {
DecodedWasm::WitPackages(..) => {
bail!("expected a component, found a WIT package")
}
DecodedWasm::Component(resolve, world_id) => (resolve, world_id),
};

let mut import_ids: Vec<String> = vec![];
for (_, import) in &resolve.worlds[world_id].imports {
if let WorldItem::Interface { id, .. } = import {
if let Some(id) = resolve.id_of(*id) {
import_ids.push(id);
}
}
}

import_ids
};
let matches = |prefix: &str| imports.iter().any(|i| i.starts_with(prefix));

if !matches("wasi:cli/environment") {
self.env = None;
}
if !matches("wasi:filesystem/") {
self.fs = None;
}
if !(matches("wasi:cli/std") || matches("wasi:cli/terminal")) {
self.stdio = None;
}
if !matches("wasi:cli/exit") {
self.exit = None;
}
if !matches("wasi:clocks/") {
self.clocks = None;
}
if !matches("wasi:http/") {
self.http = None;
}
if !matches("wasi:sockets/") {
self.sockets = None;
}
if !matches("wasi:random/") {
self.random = None;
}

Ok(())
}
None => bail!("filtering imports can only be applied to composed components"),
}
}

pub fn finish(&mut self) -> Result<VirtResult> {
let mut config = walrus::ModuleConfig::new();
config.generate_name_section(self.debug);
Expand Down Expand Up @@ -299,10 +364,37 @@ impl WasiVirt {

// now adapt the virtualized component
let encoder = ComponentEncoder::default().validate(true).module(&bytes)?;
let encoded = encoder.encode()?;
let encoded_bytes = encoder.encode()?;

let adapter = if let Some(compose_path) = &self.compose {
let compose_path = PathBuf::from(compose_path);
let dir = env::temp_dir();
let tmp_virt = dir.join(format!("virt{}.wasm", timestamp()));
fs::write(&tmp_virt, encoded_bytes)?;

let composed_bytes = ComponentComposer::new(
&compose_path,
&wasm_compose::config::Config {
definitions: vec![tmp_virt.clone()],
..Default::default()
},
)
.compose()
.with_context(|| "Unable to compose virtualized adapter into component.\nMake sure virtualizations are enabled and being used.")
.or_else(|e| {
fs::remove_file(&tmp_virt)?;
Err(e)
})?;

fs::remove_file(&tmp_virt)?;

composed_bytes
} else {
encoded_bytes
};

Ok(VirtResult {
adapter: encoded,
adapter,
virtual_files,
})
}
Expand Down Expand Up @@ -347,3 +439,10 @@ fn apply_wasm_opt(bytes: Vec<u8>, debug: bool) -> Result<Vec<u8>> {
Ok(bytes)
}
}

fn timestamp() -> u64 {
match SystemTime::now().duration_since(SystemTime::UNIX_EPOCH) {
Ok(n) => n.as_secs(),
Err(_) => panic!(),
}
}
18 changes: 18 additions & 0 deletions tests/cases/encapsulate-component.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
component = "do-everything"
compose = true

[virt-opts]
clocks = true
# http will be filtered out because the "do-everything" component doesn't import it
http = true
stdio.stdin = "ignore"
stdio.stdout = "ignore"
stdio.stderr = "ignore"

[expect.imports]
required = [
"wasi:clocks/wall-clock",
]
disallowed = [
"wasi:http/incoming-handler",
]
Loading

0 comments on commit c451261

Please sign in to comment.