unable to import or require 0.7.1 under -r esm

[warner]

All of the agoric-sdk code is still using -r esm (we use import and export as if we were writing modules, but the package.json do not declare them as such, and we rely upon -r esm to let us load module-ish code anyways).

I tried to import ses as usual, and got an error which I don't know how to work around. The minimal test case looks like:

warner@turing:/tmp$ mkdir t
warner@turing:/tmp$ cd t
warner@turing:/tmp/t$ yarn add ses@0.7.1 esm
yarn add v1.22.0
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 3 new dependencies.
info Direct dependencies
├─ esm@3.2.25
└─ ses@0.7.1
info All dependencies
├─ @agoric/make-hardener@0.0.6
├─ esm@3.2.25
└─ ses@0.7.1
Done in 0.39s.
warner@turing:/tmp/t$ cat >t.js
import { lockdown } from 'ses';
warner@turing:/tmp/t$ node -r esm t.js
/tmp/t/node_modules/ses/src/lockdown-shim.js:1
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /tmp/t/node_modules/ses/src/lockdown-shim.js
require() of ES modules is not supported.
require() of /tmp/t/node_modules/ses/src/lockdown-shim.js from /tmp/t/t.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename lockdown-shim.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /tmp/t/node_modules/ses/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:13) {
  code: 'ERR_REQUIRE_ESM'
}

I know that -r esm is a crutch, and that it doesn't play nicely with real modules: I compiled a list of rules in Agoric/agoric-sdk#527 (comment) , and I think we're running afoul of at least one of them here. But I'm not prepared to change all of agoric-sdk yet, it's a big project. I was hoping the boundary between agoric-sdk and SES-shim would let me avoid doing that.

I think esm is rewriting my t.js into CommonJS format, into something like const { lockdown } = require('ses');. If I write that into t2.js, I get a slightly different error:

warner@turing:/tmp/t$ cat >t2.js
const { lockdown } = require('ses');
warner@turing:/tmp/t$ node t2.js
internal/modules/cjs/loader.js:1167
      throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);
      ^

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /tmp/t/node_modules/ses/dist/ses.cjs.js
require() of ES modules is not supported.
require() of /tmp/t/node_modules/ses/dist/ses.cjs.js from /tmp/t/t2.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename ses.cjs.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /tmp/t/node_modules/ses/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:13)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)
    at Module.require (internal/modules/cjs/loader.js:1040:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/t/t2.js:1:22)
    at Module._compile (internal/modules/cjs/loader.js:1151:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1171:10)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14) {
  code: 'ERR_REQUIRE_ESM'
}

Note that t.js failed as it tried to require ses/src/lockdown-shim.js (which is listed in ses 's package.json as both main: and exports.import), while t2.js failed as it tried to require dist/ses.cjs.js (which is listed as exports.require).

If I run node -r esm t2.js, I get the same error as node -r esm t.js: unable to require src/lockdown-shim.js.

Any ideas?

[warner]

Interesting: if I rename node_modules/ses/dist/ses.cjs.js to ses.cjs.cjs, and update the exports.require key in node_modules/ses/package.json to match, then node t2.js works without problems. I think having it be named *.js in a package with type: 'module' tells node to treat it as a module, when it's clearly CommonJS format.

This doesn't help me use module-style imports yet, though.

[warner]

Changing SES's package.json to say main: './dist/ses.cjs.cjs' and leaving exports.require alone (and leaving it at type: 'module') enables me to run node -r esm t.js (the form that does import { lockdown } from 'ses').

[warner]

My best situation so far is when SES's package.json says:

type: 'module',
main: './dist/ses.cjs.cjs',
exports: {
 import: './src/lockdown-shim.js',
 require: './dist/ses/cjs.cjs',
 browser: './dist/ses.umd.js',
}

in which case:

• node t2.js (the all-CJS case) works, with a warning about "Conditional exports is an experimental feature", including a call to lockdown()
• node -r esm t.js (i.e. import 'ses') works without any warnings, including a call to lockdown()
• node t.mjs (same code but different suffix) loads, warns about both ESM loader and conditional exports, but actually using lockdown() runs into lockdown() throwing TypeError in 0.7.1 #213

[warner]

For reference, https://github.com/nodejs/node/blob/master/doc/api/esm.md#conditional-exports describes how exports is supposed to work. I wonder if -r esm only pays attention to main and not to exports.import.

[kriskowal]

Fixing with #217

[kriskowal]

We’ve addressed this issue with a cocktail of adjustments to how leaves on the dependency graph provide hybrid support for CJS, RESM, and ESM.

rollup config:

1. generate a dist .cjs file. The extension must be .cjs, not .cjs.js. Node.js would treat it as ESM if it were .js and package.json said type:module)
2. generate a dist .umd.js file.

package.json:

1. "type" is "module" (so Node.js will treat .js as .mjs rather than .cjs)
2. "main" must point to dist/name.cjs (so older versions of Node.js see CommonJS)
3. "module" must point to src/main.js (so tools like RESM use this instead of main)
4. "browser" must point to dist/name.umd.js (so tools like Rollup use this instead of main)
5. "exports.import" must point to src/main.js (so new Node.js versions use the ESM form)
6. "exports.require" must point to dist/name.cjs
7. "exports.browser" must point to dist/name.umd.js

This seems to cover all the bases.