feat(swingset): allow vats to be defined by a buildRootObject export

Previously, vat source files had to provide a default export function which
is invoked as `setup()`.

Now, in addition to that legacy mode, they can export a named function
`buildRootObject`, which is called with a single `vatPowers` argument.

Adds docs to describe how vats are defined, and what their code can expect.

This doesn't change any vat definitions yet, it just adds the nde
`buildRootObject` mode.

packages/SwingSet/docs/static-vats.md

@@ -0,0 +1,106 @@
+# Static Vats
+
+A SwingSet machine has two sets of vats. The first are the *Static Vats*, which are defined in the configuration object, and created at startup time. The root objects of these vats are made available to the `bootstrap()` method, which can wire them together as it sees fit.
+
+The second set are the *Dynamic Vats*. These are created after startup, with source code bundles that were not known at boot time. Typically these bundles arrive over the network from external providers. In the Agoric system, contracts are installed as dynamic vats (each spawned instance goes into a separate vat).
+
+This document describes how you define and configure the static vats.
+
+## Creating a Static Vat
+
+The source code for all static vats must be available at the time the host application starts up. This source code, and its dependencies, must not change for the lifetime of the SwingSet machine (which, through persistence and the kernel state database, extends beyond a single process). Applications are encouraged to copy the static vat sources into a working directory during some sort of "init" step, where they'll remain untouched by normal development work on the original files. But note that this won't protect against changes in dependencies.
+
+Static vats are defined by a JS module file which exports a function named `buildRootObject`. The file may export other names; these are currently ignored. The module can import other modules, as long as they are pure JS (no native modules or binary libraries), and they are compatible with SES. See `vat-environment.md` for details of the kind of JS you can use. The static vat file will be scanned for its imports, and the entire dependency graph will be merged into a single "source bundle" object when the kernel first launches.
+
+The `buildRootObject` function will be called with one object, named `vatPowers`. The contents of `vatPowers` are subject to change, but in general it provides pure functions which are inconvenient to access as imports (such as `tildotTransform`), and vat-specific authorities that are not easy to express through syscalls (such as control over local metering). See below for the current property list.
+
+`buildRootObject` is expected to return a hardened object with callable methods and no data properties (note that `harden` is available as a global). For example:
+
+```js
+export function buildRootObject(vatPowers) {
+  let counter = 0;
+  return harden({
+    increment() {
+      counter += 1;
+    },
+    read() {
+      return counter;
+    }
+  });
+}
+```
+
+Each vat has a name. A *Presence* for this vat's root object will be made available to the bootstrap function, in its `vats` argument. For example, if this vat is named `counter`, then the bootstrap function could do:
+
+```js
+function bootstrap(argv, vats, devices) {
+  vats.counter~.increment();
+}
+```
+
+The *bootstrap function* is the method named `bootstrap` on a special *bootstrap vat*. This is a static vat defined in a separate section of the config object, currently named `config.bootstrapIndexJS`. This vat is given access to the root objects of all other vats, as well as access to all devices, plus a set of arguments passed into `buildVatController`, all delivered in the bootstrap message. This message is synthesized by the kernel and sent automatically at machine startup. Since no other messages exist at startup time, and only connectivity begets connectivity (as befits an object-capability system), the bootstrap function is entirely responsible for establishing the inter-vat connections necessary for subsequent activity.
+
+### Legacy setup() Function
+
+More generally, vats are defined in terms of a `syscall` object (for the vat to send instructions into the kernel), and a `dispatch` object (for the kernel to send messages into the vat). Vats can provide a `setup()` function which is given a `syscall`, among other things, and are expected to return a `dispatch`. Vats defined this way are not obligated to provide object-capability security within the vat. For example, `syscall.send()` can be used to send a message to any remote object that has ever been granted to anything within the vat. If everything within the vat can reach `syscall`, then every object within the vat is as powerful as any other, and there is no partitioning of authority within the vat.
+
+Most vats use "liveslots", a layer which provides an object-capability environment within a vat. Liveslots is a library which provides a `makeLiveslots()` function. Vats which use liveslots will export a `setup()` function which calls `makeLiveslots()` internally. These vats tend to have boilerplate like this:
+
+```js
+export default function setup(syscall, state, helpers, vatPowers0) {
+  return helpers.makeLiveSlots(syscall, state,
+                               (E, D, vatPowers) => buildRootObject(vatPowers),
+                               helpers.vatID,
+                               vatPowers0);
+}
+```
+
+These vats are still supported, for now. Any vat source file which exports `buildRootObject()` will automatically use liveslots. If the file does *not* export `buildRootObject()`, it is expected to export a `default` function that behaves like the `setup()` described above.
+
+A few vats do not use liveslots. The main one is the "comms vat", which performs low-level mapping of kernel-sourced messages into strings that are sent off-machine to other swingsets. This mapping would be rather inefficient if it went through the serialization/deserialization layers that liveslots provides to normal vats. The comms vat will eventually be loaded with some special configuration flag to mark it as non-liveslots, rather than retaining the `default`/`setup()` fallback.
+
+## VatPowers
+
+Static vats currently receive the following objects in their `buildRootObject()`'s sole `vatPowers` argument:
+
+* `Remotable`
+* `getInterfaceOf`
+* `makeGetMeter`
+* `transformMetering`
+* `transformTildot`
+
+(dynamic vats do not get `makeGetMeter` or `transformMetering`)
+
+### marshalling: Remotable and getInterfaceOf
+
+Messages sent between vats include arguments (and return values) which are serialized with `@agoric/marshal`. This marshalling library makes a distinction between plain data, and "remotable objects". Data is merely copied, but remotables arrive as a *Presence*, to which messages can be sent, with e.g. `counter~.increment()`.
+
+Objects which are frozen, and whose enumerable properties are all functions, will automatically qualify as remotable. In addition, vats can use `Remotable()` to create new remotable objects with a particular "interface" name, as well as other properties. The interface name can be retrieved with `getInterfaceOf`.
+
+This functionality is still in development. See https://github.com/Agoric/agoric-sdk/issues/804 for details.
+
+### metering: makeGetMeter, transformMetering
+
+Static vats are, for now, allowed to apply metering enforcement within their own walls. These vat powers provide the tools they need to do this. `makeGetMeter` returns a new `getMeter` function and a collection of functions to check and refill the meter it wraps. `transformMetering` takes a string of code and returns a new string into which metering code has been injected. By including `getMeter` in the `endowments` of a new `Compartment`, and adding `transformMetering` in the `transforms` of that compartment, vats can impose metering limits on other code evaluated inside the new compartment. When the meter runs out, the code subject to that meter starts throwing exceptions (which it cannot catch itself), and keeps throwing them until the meter is refilled.
+
+This approach is not yet complete or sound (the evaluated code may be able to consume more CPU or memory than the meter ought to allow), and it is easy for the vat code to get confused about how much progress it has made. The metered code might call back into the vat's code, and cause a metering exception to happen halfway through that call, leaving the vat in a confused state. The safest way to meter code is to run it in a completely separate (dynamic) vat, which will live or die as a single unit.
+
+### wavy-dot: transformTildot
+
+The "tildot transform" converts wavy-dot (aka tildot) syntax (`counter~.increment()`) into invocations of `HandledPromise` APIs (`HandledPromise.applyMethod(counter, "increment", [])`). The function passed as `vatPowers.transformTildot` accepts one string and returns a new string, with the transform applied.
+
+This is particularly useful for vats that implement a REPL, so operators can include tildot syntax in the expressions they submit. Such a vat will have to transform each expression string before passing it to e.g. `compartment.evaluate()`, generally by creating a new `Compartment` with `transforms: [ vatPowers.transformTildot ]`.
+
+`transformTildot` is provided in `vatPowers` because it relies upon the Babel parser library, and Babel does not run inside a SES non-"Start Compartment" yet. Also, Babel has a lot of code, and bundling all of it into a vat would make the vat bundle file pretty large. Finally, `transformTildot` is unmetered. This doesn't matter for static vats (which aren't metered either), but for dynamic vats (which *are* metered), calling `transformTildot` under metering would consume a lot of the vat's metering budget.
+
+## Configuring Vats
+
+Each swingset is created by a call to `buildVatController`, which takes a `config` argument. The `config.vats` property is a Map of named vat definitions: each value is an object with `sourcepath` and `options`. The `sourcepath` is the filename of the vat definition file (the one that exports `buildRootObject`).
+
+The `options` bag currently only recognizes one property: `enablePipelining`. When a pipelining-enabled vat is the *decider* of some unresolved promise, and some other vat sends a message to that promise, the kernel will deliver the message to the vat, rather than holding the message in the kernel queue until the promise becomes resolved. Pipelining is enabled on the comms vat, where it is vital to reduce latency, but disabled on all other vats, where it wouldn't help very much.
+
+## Built-in Vats
+
+The kernel has a handful of built-in vats, which are automatically added and do not need to appear in the `config.vats` table. One is used to create new dynamic vats, this is known as the `vatAdmin` vat.
+
+Other vats are defined by swingset but must be added (to `config.vats`) by the application. These include the timer management vat and the comms vat. These vats might be turned into built-in vats in the future, and removed from the application's sphere of responsibility.

packages/SwingSet/src/controller.js

@@ -195,7 +195,20 @@ export async function buildVatController(config, argv = []) {
       filePrefix: name,
       endowments: vatEndowments,
     });
-    const setup = vatNS.default;
+    let setup;
+    if ('buildRootObject' in vatNS) {
+      setup = (syscall, state, helpers, vatPowers) => {
+        return helpers.makeLiveSlots(
+          syscall,
+          state,
+          (_E, _D, vatP) => vatNS.buildRootObject(vatP),
+          helpers.vatID,
+          vatPowers,
+        );
+      };
+    } else {
+      setup = vatNS.default;
+    }
     return setup;
   }
 

packages/SwingSet/src/kernel/kernel.js

@@ -415,6 +415,12 @@ export default function buildKernel(kernelEndowments) {
     kernelPanic = new Error(`kernel panic ${problem}`);
   }
 
+  // returns a message-result reader, with .status() (that returns one of
+  // 'pending', 'fulfilled', or 'rejected') and .resolution() (that returns a
+  // value only if status is not 'pending')
+  //
+  // 'policy' is one of 'ignore', 'logAlways', 'logFailure', or 'panic'
+  //
   function queueToExport(vatID, vatSlot, method, args, policy = 'ignore') {
     // queue a message on the end of the queue, with 'absolute' kernelSlots.
     // Use 'step' or 'run' to execute it

packages/SwingSet/test/definition/test-vat-definition.js

@@ -0,0 +1,66 @@
+/* global harden */
+import '@agoric/install-ses';
+import tap from 'tap';
+import { buildVatController } from '../../src/index';
+
+const mUndefined = { '@qclass': 'undefined' };
+
+function capdata(body, slots = []) {
+  return harden({ body, slots });
+}
+
+function capargs(args, slots = []) {
+  return capdata(JSON.stringify(args), slots);
+}
+
+tap.test('create with setup and buildRootObject', async t => {
+  const config = {
+    vats: new Map(),
+  };
+  config.vats.set('setup', {
+    sourcepath: require.resolve('./vat-setup.js'),
+    options: {},
+  });
+  config.vats.set('liveslots', {
+    sourcepath: require.resolve('./vat-liveslots.js'),
+    options: {},
+  });
+  const c = await buildVatController(config, []);
+  let r = c.queueToVatExport('setup', 'o+0', 'increment', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(r.resolution(), capargs(mUndefined), 'setup incr');
+  r = c.queueToVatExport('setup', 'o+0', 'read', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(r.resolution(), capargs(1), 'setup read');
+  r = c.queueToVatExport('setup', 'o+0', 'tildot', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(
+    r.resolution(),
+    capargs('HandledPromise.applyMethod(x, "foo", [arg1]);'),
+    'setup tildot',
+  );
+  r = c.queueToVatExport('setup', 'o+0', 'remotable', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(
+    r.resolution(),
+    capargs('iface1'),
+    'setup Remotable/getInterfaceOf',
+  );
+
+  r = c.queueToVatExport('liveslots', 'o+0', 'increment', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(r.resolution(), capargs(mUndefined), 'ls incr');
+  r = c.queueToVatExport('liveslots', 'o+0', 'read', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(r.resolution(), capargs(1), 'ls read');
+  r = c.queueToVatExport('liveslots', 'o+0', 'tildot', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(
+    r.resolution(),
+    capargs('HandledPromise.applyMethod(x, "foo", [arg1]);'),
+    'ls tildot',
+  );
+  r = c.queueToVatExport('liveslots', 'o+0', 'remotable', capargs([]), 'panic');
+  await c.run();
+  t.deepEqual(r.resolution(), capargs('iface1'), 'ls Remotable/getInterfaceOf');
+});

packages/SwingSet/test/definition/vat-liveslots.js

@@ -0,0 +1,20 @@
+/* global harden */
+
+export function buildRootObject(vatPowers) {
+  let counter = 0;
+  return harden({
+    increment() {
+      counter += 1;
+    },
+    read() {
+      return counter;
+    },
+    tildot() {
+      return vatPowers.transformTildot('x~.foo(arg1)');
+    },
+    remotable() {
+      const r = vatPowers.Remotable('iface1');
+      return vatPowers.getInterfaceOf(r);
+    },
+  });
+}

packages/SwingSet/test/definition/vat-setup.js

@@ -0,0 +1,30 @@
+/* global harden */
+
+function buildRootObject(vatPowers) {
+  let counter = 0;
+  return harden({
+    increment() {
+      counter += 1;
+    },
+    read() {
+      return counter;
+    },
+    tildot() {
+      return vatPowers.transformTildot('x~.foo(arg1)');
+    },
+    remotable() {
+      const r = vatPowers.Remotable('iface1');
+      return vatPowers.getInterfaceOf(r);
+    },
+  });
+}
+
+export default function setup(syscall, state, helpers, vatPowers0) {
+  return helpers.makeLiveSlots(
+    syscall,
+    state,
+    (E, D, vatPowers) => buildRootObject(vatPowers),
+    helpers.vatID,
+    vatPowers0,
+  );
+}