Modules and Phasing

Issue #223 (closed).

This is not ready to be merged just yet but I’ve been making some good progress on getting proper phasing and module support working and wanted to give people a heads up on what is coming.

The approach I’m taking is heavily modeled after how Racket does it as described in Composable and Compilable Macros. This means:

Explicit Phasing

Import statements declare which phase they want to import into. So:

import { m } from 'm';  // bound for runtime
import { n } from 'n' for syntax;  // bound for compiletime
m;
n; // unbound
syntax s = function (ctx) {
    m; // unbound
    n;
    // ...
}

Multiple Instantiation

There is a tower of phases (a module that is import ... for syntax might have a import ... for syntax which might have a import ... for syntax which might have…) and a module is separately instantiated in each phase. This is implemented via node’s vm.runInContext.

Common Concerns

Unfortunately there are at least four different kinds of modules we need to support:

• CommonJs modules (require and exports)
• node compiled modules (.node)
• ES2015 modules
• Sweet modules (ES2015 with syntax definitions)

Obviously Sweet modules are the easiest to handle, just compile and then invoke or visit as appropriate. We can recognize Sweet modules with the #lang pragma and then use the standard pipeline.

Dealing with the other kinds of modules is more tricky.

ES2015 modules are conceptually straightforward (just compile like normal) but knowing that a file is ES2015 is complicated right now because the community hasn't settled on a standard way to do that. There is an ongoing node discussion where it sounds like they are converging on a file extension (sigh which is wrong, I think rollup got it right by using package.json, but whatever). Anyway, it's not settled yet.

The important one is commonjs since at the moment almost everything is commonjs. I think it is straightforward to just plop a commonjs module inside a vm.runInContext with hooks into the exports object but I haven’t tried yet.

I’m pretty sure we can’t do anything about .node compiled modules besides requireing them. This is probably fine?

/cc @elibarzilay @natefaubion