Replies: 4 comments 1 reply
-
#5248 is an example internal version publish conflicts. |
Beta Was this translation helpful? Give feedback.
-
I think the biggest change for this one is proc_macro based packages, if they need to be exposed as public interface. As our goal is make users to avoid to reference separate packages, proc_macro packages need to reference swc_core:: 's reexported one instead of direct reference. This is already done for the plugins (i.e swc_plugin_macro always refer swc_plugin's exported, instead of referencing things directly) but other packages doesn't: may need some refactorings to make sure this work. The other aspect is how do we have reliable tests to verify new PR / commit doesn't cause breaking changes. Other than having some simple compile-time codes, not sure if we can have extensive tests. Even removing features can be a breaking change. |
Beta Was this translation helpful? Give feedback.
-
I'll be changing this to @kdy1 - thoughts, concerns, or objections? |
Beta Was this translation helpful? Give feedback.
-
So we did some discussions outside the RFC itself, summarizing here for the visibility. Each swc_* package currently provides some sort of semver, but the major collision comes from inter-package version conflict. For the simplified example it looks like this:
Depends on the compatibility of x, x+1, x+2, it may work or not - but users don't have a good way to determine which version of swc_common they need to import, since swc_plugin doesn't transparently expose transitive dep's version / and also swc/core does not have a way to determine which version is compatible. RFC tried to amend this by all 3 using single pkg (swc_core), and swc_core itself manages inter-pkg version to avoid this kind of conflict. So far, we agreed to move this into |
Beta Was this translation helpful? Give feedback.
-
Summary
Provides compile time, runtime compatibility across various hosts and plugins compiled against different versions of
swc_*
packages.Motivation
Currently, there are a handful of packages (or rusty term
crates
) published to allow to use SWC's core features. Major usecases for this packages are either creating a custom, native binary instead of using@swc/core
to apply customizations (i.e https://nextjs.org/docs/advanced-features/compiler), or writing a plugin for the@swc/core
which is a rust based codes requires to import certain feature of SWC.Either you want to write a custom binary or a plugin, eventually this is typical examples of defining dependencies for the features of SWC. Each
swc_*
package intended to be a micro size module serves specific public interfaces to use. So there is no good way to avoid having multiple dependencies. Even though some dependencies are being used as transitive dependency - for example, pretty much most packages have reference toswc_common
- if an application / plugin needs to import something directly fromswc_common
it has to be listed as direct dependency. The problem begins from here.Unlike node.js bindings (
@swc/core
), any rust packages do not honor semver and there is no clear guarantee certain version is compatible to others. Instead, we follow the moving forward model; the only thing we try to ensure is `if all the swc_* packages are latest, it should be built. While this wasn't cleanly understandable, it worked for most cases until we started to face cases we couldn't ensure those rules.First, there are some chances certain transitive dependency's out of sync, so it conflicts with direct dependencies version.
This one is more close to the bug actually, since SWC's publish should take care of bump up all the transitive dependencies altogether. But in reality this happens, and it happens more frequently as we move fast.
The second problem is bigger and not easily able to work around. If host & plugins have incompatible versions between 2, it'll cause a conflict either in build or runtime without clear message.
This is a simplified example of the version differences. This may work or not depends on the changes between 0.63 and 0.64. For this simplified example, the only dependency is
swc_plugin
which can be resolved easily. If a plugin gets complex and trying to involve more dependencies it exposes more collision points such asswc_plugin
to others. Issues like swc-project/plugins#48 is caused by this.More headache is to deal with multiple host runtimes.
next/swc
is a variant of SWC, which does not always have the same version / feature set as@swc/core
. If those 2 versions are incompatible, there's no way to make the plugin run on both hosts, or not even able to build it from the beginning (https://github.com/vercel/next.js/issues/38428).One of the main goals of the SWC's plugin system is having a single binary (wasm), runs on the supported platform as long as
@swc/core
reaches a major breaking change. With the current versioning system, it is not possible to achieve.Lastly, there's a known discoverability issue as well. Since features are scattered around
swc_*
, it is unclear which package to be used for certain features. This isn't new and partially resolved by documentation though.Detailed design
To avoid version conflicts across different packages, propose a new meta package
swc_core
. Package name is tentative, subject to change.swc_core
attempts to be served as isomorphic entry point to any features of SWC to the consumers. For writing custom native binary, or plugin, both needs to specifyswc_core
as a single dependency. It is a meta package that does not contain code on its own but only tries to reexport from transitive swc_* dependencies. Reexports are controlled by cargo's feature to avoid bringing in all the codes into compiled binary.Once it's completed,
swc_core
will try to honor semver. If there are any changes that different binaries rely on different versions ofswc_core
causes regression, it'll be considered breaking changes. Any other remaining packages will be treated as same. SWC will not prohibit the use of individual packages, but it can break anytime each time a new version's released user needs to resolve manually.Drawbacks
This will bring interesting challenges to manage versions across dependencies to be reexported by meta package. Also, this creates a mega package that brings all the features into a single package, which potentially be a bottleneck for the build performance as well as raising tricky puzzles between non-compatible features as similar to plugin's wasm runtime.
Alternatives
We could try to ensure each individual package is compatible across versions. But this will be a more tricky challenge to verify various dependency combinations with possible transitive dependency conflicts.
Unresolved questions
There are some open questions that need to be resolved.
proc_macro
and direct dependenciesIt is unclear if we can prevent any additional imports to deps if proc_macro is involved. Similar to
swc_plugin_macro
probably we could make macro impl relies on the namespace of swc_core always. But this will be a large scale refactoring that requires careful approach to not break existing codebases.There is no good known deterministic testing we can say a new version is compatible to the old version.
There are numbers of features exposed by each individual package.
swc_core
needs a way to expose it to public interface user can control. As there are a lot of tangled dependency trees internally, properly forwarding a specific feature from top to bottom may require sizable changes to existing feature / codebase.References (if exists)
#5149
Implementations
Beta Was this translation helpful? Give feedback.
All reactions