This is SWIG JavaScript Evolution, an up-to-date fork of the SWIG project with a focus on JavaScript - native/Node.js, WASM/Node.js and WASM/browser.
-
Interfacing with Node.js entirely through Node-API
- Allowing for the distribution of universal pre-built binary modules, independent of the Node.js version, through npm - ie, your user types
npm install
and can start using your C++ library a few seconds later without having a build environment worker_threads
support out of the box- Automatic support for alternative Node.js derived frameworks such as Electron (This part of SWIG JSE has already been merged in the mainline SWIG 4.2.0)
- Allowing for the distribution of universal pre-built binary modules, independent of the Node.js version, through npm - ie, your user types
-
Automatic support for multi-threaded asynchronous execution for all C++ code through a simple flip switch One of the weakest points of Node.js in particular, and JavaScript in general, is its multithreading model. SWIG JSE changes this, bringing powerful multithreading support to developers coming from the Web world with no previous parallel programming experience. All locking is automatic and transparent and it includes deadlock prevention. Instead of writing:
const result = nativeModule.mathHeavyStuff();
your user writes:
const result = await nativeModule.mathHeavyStuffAsync();
and the code automatically runs in a separate background thread. Both CPU-heavy and I/O-heavy operations become automatically asynchronous. (This part of SWIG JSE is currently being considered for merging in the mainline SWIG 4.2.0)
-
TypeScript Support
Since its inception, TypeScript has surfed the JavaScript wave and after several iterations, it is becoming the language of choice for JavaScript development in the enterprise world - because of the additional coding discipline that it imposes. Modules produced by SWIG JSE support both JavaScript and TypeScript - and are fully typed when used from TypeScript - imposing the additional checks that are expected of TypeScript developers.
-
WASM Support
The true reason for the mass adoption of JavaScript is the fact that to this date, it remains the single language supported natively by web browsers. As more and more software moves to the web, there is a huge demand for porting legacy C/C++ software to this new platform. WASM is the answer to this demand, but porting C/C++ software to JavaScript also requires writing a huge amount of glue code - which is also the main object of SWIG. Thus, naturally, SWIG also evolves towards this new platform.
Just like JavaScript offers the unique possibility of directly sharing code between the front-end (the browser) and the back-end (usually Node.js, but now there are alternatives), so SWIG JSE offers the unique possibility of writing an universal JavaScript wrapper, which can be compiled to a native Node-API addon for Node.js and to a WASM binary for the browser - with the two having identical APIs and sharing every single bit of code.
A first project,
magickwand.js
has been published tonpm
as a testament to the capabilities offered by this new tool. It features the full ImageMagick-7 library - including support for multithreaded operations throughasync
/await
and TypeScript bindings - generated by about 700 lines of SWIG code that produces 400K lines of C++ code. Both the Node.js and the browser versions supportasync
/await
with an absolutely identical interface.Unlike all other supported languages, SWIG WASM does not target native code - on this platform JavaScript is the native language and the C/C++ code is the exotic environment. SWIG WASM is made possible by
emnapi
which provides a Node-API interface for interfacing with the JS engine in the browser. -
Code Splitting
Although this feature has been mentioned many times in the past, the memory requirements of WASM compilation make it mandatory. SWIG JSE can produce separate C++ headers and multiple compilation units. At the moment only JavaScript Node-API and Javascript WASM can use this feature, but it is planned to eventually extend it to Python.
SWIG JavaScript Evolution is currently available only as a build from source from this repository and as a Github Action.
You can find an example skeleton for a new project that uses a dual-build Node.js/native and browser/WASM system at swig-napi-example-project
.
If you want to see a real-world complex project that uses conan
to manage a large pool of dependencies - including WASM builds - you should take a look at magickwand.js
- the ImageMagick-7 bindings for JavaScript.
I am currently working on a second generation C++/JavaScript project with a new build system which will be based on the meson
/conan
/xpm
trio instead of node-gyp
- it will feature clean integration of 3rd party C++ libraries on all platforms (conan
), use of the native build system (meson
) and fully self-contained builds that do not need a working C++ environment on the final user host (xpm
). This new project is the PROJ
bindings for JavaScript. The project is partially functional and still not published.
# from git
git clone https://github.com/mmomtchev/swig.git
# from a release
tar -zxvf v5.0.0.tar.gz
cd swig
# add a suffix if you want to be able to use the main trunk SWIG
# on the same host
./configure --program-suffix=-jse
# this will install everything in /usr/local/share/swig-jse
# w/o affecting your existing SWIG
make -j 4 && sudo make install
For WASM, you will also need to install emsdk
and setup its environment variables as described at
https://github.com/emscripten-core/emsdk.
SWIG JSE is now available on conan
. You can simply add it to your conanfile.txt
:
[tool_requires]
swig-jse/[>=5.0.3]
# optionally, get also emscripten
emsdk/[>=3.1.50]
It is available on my custom conan
repository:
conan remote add swig-jse https://swig.momtchev.com/artifactory/api/conan/swig-jse
You can either use the prebuilt packages - which should now work on all three major OS - or - should you be unsettled by the recent events around xz-utils
- you can also build it yourself.
The conan recipes are available at https://github.com/mmomtchev/swig-conan.
The only changes in the manual relative to the mainline SWIG are in the JavaScript section.
Late technology preview / Early public release. The first open-source package - magickwand.js
- is published on npm
and it is used in production on some low-traffic websites. It uses all major new SWIG JSE features - async, WASM, locking and TypeScript. A second one, proj.js
is currently in the works.
Currently, WASM projects using asynchronous wrappers require that COOP
/COEP
is enabled. In this example, it is enabled by the webpack
built-in server and by the karma
test runner. Users of your module will have to host it on web servers that support and send these headers - this is a requirement on the web server end - ie a configuration option that must be enabled in Apache or nginx. For example, currently Github Pages and many low-end hosting providers do not support it.
Alternatively, this example can be built without asynchronous wrappers in order to produce a WASM binary that does not require COOP
/COEP
. The only real difference is the emscripten
build configuration which can be found in emscripten.gypi
.
In this case, there are two possible strategies:
- Accept that calling C++ functions will produce main thread latency - which works well if all your C++ methods run very fast
- Use
GoogleCromeLabs/comlink
to call them in a worker thread - which works well if all your C++ methods have very long execution times because it adds significant overhead when calling them (this will require a custom serializer for C++ object - I plan to make an example)
Mixing the two is possible, but C++ functions running in the main thread and C++ functions running the in comlink
worker won't be able to share objects as they will be running in separate memory spaces.
SWIG-generated projects for JavaScript can currently choose between two build systems:
Description | node-gyp |
meson + conan + xpm |
---|---|---|
Overview | The official Node.js and Node.js native addon build system from the Node.js core team | A new, still under development, experimental build system from SWIG JSE |
Status | Very mature | Still not completely finished |
Platforms with native builds | All platforms supported by Node.js | Linux, Windows and macOS |
WASM builds | Hackish, see swig-napi-example-project and magickwand.js@1.1 for solutions |
Out-of-the-box |
Node.js APIs | All APIs, including the now obsolete raw V8 and NAN and the current Node-API | Only Node-API |
Integration with other builds systems for external dependencies | Very hackish, see magickwand.js@1.1 for solutions, the only good solution is to recreate the build system of all dependencies around node-gyp |
Out-of-the-box support for meson , CMake and autotools |
conan integration |
Very hackish, see magickwand.js@1.0 |
Out-of-the-box |
Build configurations through npm install CLI options |
Yes | Yes |
Distributing prebuilt binaries | Yes, multiple options, including @mapbox/node-pre-gyp , prebuild-install and prebuildify |
prebuild-install |
Requirements for the target host when installing from source | Node.js, Python and a working C++17 build environment | Only Node.js when using xpack-dev-tools , a working C++17 build environment otherwise |
Makefile language | Obscure and obsolete (gyp ) |
Modern and supported (meson ) |
When choosing a build system, if your project:
-
targets only Node.js/native and has no dependencies
→ stay on
node-gyp
-
meant to be distributed only as binaries compiled in a controlled environment
→ stay on
node-gyp
-
has a dual-environment native/WASM setup
→
node-gyp
will work for you, buthadron
has some advantages -
has dependencies with different build systems (
meson
,CMake
,autotools
)→
hadron
is the much better choice -
uses
conan
→
hadron
is the much better choice -
everything at once
→
hadron
is the only choice
You can find the classical SWIG Node-API project skeleton using node-gyp
here. It has a dual-environment setup with Node.js/native and Browser/WASM builds.
You can find the new SWIG Node-API project skeleton using hadron
here. It has a dual-environment setup with Node.js/native and Browser/WASM builds and external dependencies using conan
.
Alas, the current state of my affair has made working with conan
and meson
extremely difficult.
In both projects, they tried to play a psychosis game with my work (my PRs) by doing simultaneously seemingly random acts and then trying to send me to see a psychiatrist. Given the context of the current affair, this has made any real work extremely difficult.
At the moment, both software packages need to be installed from my own repositories.