React Native “Plugins” for Native Libraries

[fkgozali]

Background

The current system to register and load React Native NativeModules is not consistent across different platforms today, and within each platform there are more than one ways to do so. This creates inconsistent runtime behaviors depending on which method is used, including the startup performance of the React Native infra (bridge). For instance, in iOS, the system relies on a lot of runtime class and protocol lookups, while in Android, one has to explicitly list out the modules they need.

In order to unify this mechanism we need to standardize the registration process so that:

• The registration is explicit, no more runtime discovery magic
• The registration doesn't add into React Native startup cost: modules should be lazily loaded by default
• The build system can statically fail early if any module accessed from JS is not available in the app

The Problem with iOS NativeModules Registration TODAY

• By default iOS NativeModules system is very expensive to load:
  • RCT_EXPORT_MODULE() adds +load method to the ObjC class, which means:
    • All of these classes are loaded during the app startup, even if React Native hasn't started
    • For big apps, classes may be split into different NSBundle's, and +load unfortunately force loads all the NSBundle's, again bad for app startup
  • Further, most NativeModules get initialized when loaded, even if they're not needed yet. Note that this is addressed by TurboModule, and is out of scope for this discussion.
• The registration is implicit: if the source file gets compiled in, the module is registered and it's discovered during runtime
  • There is no explicit list of modules supported by the app, which means no one knows during build time if one forgets to install a NativeModule
  • Runtime discovery may break if the compiler optimization wipes out the symbols by removing -ObjC flag
    • This includes resolving class names from string during runtime
  • "RCT" and "RK" prefix in the class names get stripped magically
• There exists a lazy way to load these modules, but there are more than one call paths, creating inconsistent runtime behaviors.

The Problem with Android NativeModules Registration Today

• There are many variants of ReactPackage today:
  • The default one is expensive to load: each module gets initialized during React Native startup
  • The lazy version avoids expensive class loading and initialize modules lazily
• The registration can be massive
  • As an app grows the number of NativeModules, the code that lists them out gets really large and it's easy to make mistakes
  • They are all hand-written, without any build time validation (similar to iOS issue)

The “Plugin” Concept

The two main concepts we'd like to apply here are:

• Using explicit registration that can be verified statically during build time
• Avoiding giant centralized configuration, allowing each module/library to “register itself” lazily

For the sake of the discussion, let's call this system the “Native Library Plugin”, or just “plugin”.

What is A Plugin?

A plugin provides:

1. Explicit annotation in the build configuration that a library provides a certain functionality, with a given name
2. Codegen system that collects all occurrence of the plugin annotation throughout the app's build configuration, then automatically generates the lookup mechanism in native code
   1. These will be purely C functions for ObjC/C++/JNI, because they are cheap
   2. The lookup function automatically loads the code from disk, regardless of where they live (e.g. it will load the right NSBundle in iOS)
3. A build-time verification:
   1. To validate that all modules in a given list exist in the app
   2. To validate that only one module of the same identifier is provided in the entire app

Plugin Schema

To define a plugin we need a simple schema containing:

• The plugin type, e.g. the string name of the plugin
• One or more lookup keys
• One or more invocation C functions

Here's a simplified example definition using BUCK syntax for iOS NativeModule. Let's call it RCT_NATIVE_MODULE_PROVIDER_SOCKET.

RCT_NATIVE_MODULE_PROVIDER_SOCKET = plugin.Socket(
    identifier = "RCTNativeModule",
    schema = {
        "name": plugin.Type.cstring(),
        "native_class_func": plugin.Function(
            return_type = plugin.Parameter("Class"),
        ),
    },
    sorted_by = "name",
)

Note that the syntax will depend on the build system in use. With the schema defined, each library will need to define what “socket(s)” it provides.

NativeModules Example

To describe this more easily, let's use an ObjC NativeModule as an example: RCTSampleModule. Consider the following BUCK file for RCTSampleModule:

apple_library(
    name = "RCTSampleModule",
    srcs = glob(["**/*.mm"]),
    headers = glob(["**/*.h"]),
    plugins = [
        RCT_NATIVE_MODULE_SOCKET(
            name = "SampleModule",
            native_class_func = "RCTSampleModuleCls",
        ),
    ],
)

For the 3 things a plugin needs to provide:

• For (1), instead of using RCT_EXPORT_MODULE() in each class, we annotate the build config for that module:
  • The name will be SampleModule - this is the name used everywhere, including JS.
  • The class that provides it is RCTSampleModule
  • The C function that will provide the class information is Class RCTSampleModuleCls()
• For (2), the system automatically generates:
  • A C lookup function given a name
  • A C function to invoke the registered function above
• For (3), this will be build-system specific, but the system can validate that all symbols listed above are present
  • For Buck, this is in form of Buck tests, evaluating all deps on the app binary target

Then with all the codegen output, we can write this simple C lookup function for the entire NativeModules in the app:

#include "PluginsSockets.h" // codegen

Class RCTNativeModulePluginClassProvider(const char *name) {
  if (!name) {
    return nil;
  }

  // The RCTNativeModuleSocket functions are codegen output
  const auto result = `RCTNativeModuleSocket`::FindPluginWithNameValue(name);
  return result ? ``RCTNativeModuleSocket``::InvokeNativeClassFunc(*result) : nil;
}

The hosting app can simply call RCTNativeModulePluginClassProvider() and not worry about how the modules are registered. Then the RCTSampleModule.mm will implement the C function:

#import "Plugins.h"

@implementation RCTSampleModule
// ...
@end

// This function implements what's registered in the build annotation
Class RCTSampleModuleCls() {
  return RCTSampleModule.class
}

In this example, once the system finds the class RCTSampleModule by invoking RCTSampleModuleCls() via the plugin system, it knows how to initialize the NativeModule properly.

Further, for some use cases, one can get a list of registered plugins as follow:

// Inside a .mm file in the app
#import "PluginsSockets.h" // codegen

// Return all classes for all NativeModules in the app
NSArray *RCTNativeModulePluginEvaluateAllClasses() {
  NSMutableArray *modules = [NSMutableArray new];
  for (const auto &feature : RCTNativeModule::Plugins()) {
    Class klass = RCTNativeModuleSocket::InvokeNativeClassFunc(feature);
    [modules addObject:klass];
  }
  return modules;
}

React Native Core Specific Plugins

There are a few core React Native functionalities that will benefit from plugins. At least each of the following shall define its own plugin schema:

• NativeModules/TurboModules, as described above
• Fabric Native Components
  • Note that the existing ViewManagers are NativeModules, but in Fabric ViewManagers will eventually go away
• Custom image loaders
• Custom image decoders
• Custom networking handler

FAQs

Q: What do you mean by having build system specific syntax for the socket definition?
There are many ways to build an app, e.g. using Buck, CocoaPods, gradle, Xcode, etc. Each of the integrations will need its own way to define the plugin sockets. As long as the codegen output is consistent across this system, the syntax for each system can be flexible.

Q: Can a library define multiple sockets? Or is it limited to one?
Each library can define 0 or more sockets of the same type, as long as the identifiers do not collide. For example, there shall be only one definition of “SampleModule” throughout the entire app. In case there are different implementation of such modules for different apps, the build dependency graph needs to make sure only exactly 1 of “SampleModule” socket is provided for each app.

Q: Is this a hard blocker for Fabric and TurboModule projects?
No, both projects have their own fallback mechanism to load the modules/components. For the initial rollout, those fallbacks will still be used. But long term, we'd like to unify the registration with this one plugin system, deprecating these fallbacks over time. For example, the legacy way of getting a module class from its name shall be deprecated eventually.

Q: It seems like only big apps may get into the issues listed above, why have this by default then?
It is more for unifying the code paths regardless of the apps you're building. Historically we had many ways to register things, as described above, and it's not always easy to keep all mechanism up-to-date and consistent. Also, with this system, we hope that by default the mechanism is as performant as it can be.

Q: How is this used at Facebook?
Facebook has its own internal implementation of the plugin system, using BUCK, and is very FB-specific. In fact, TurboModules and Fabric components are using these system internally. This is why we need help building the same system in OSS environment so that there's only one system being used everywhere.

Q: There's already autolinking effort, will this be compatible?
Yes, in fact, this can be additional features on top of autolinking, not a complete replacement.

[matt-oakes]

I haven't fully digested this, but I have a couple of questions first:

• How does this affect current RN libraries? Will it continue to be possible to use the currently released libraries when this feature is implemented? I presume this old interface will be deprecated if so.
• If a library launches with support for the new "plugin" system, will it be possible for this new version to be installed on older versions of React Native which does not use the "plugin" system? Essentially, will a library be able to support both the old and new interface, or will they need to release a version which breaks backwards compatibility?

I think it's pretty important that old libraries continue to work and new library releases can use the "plugin" system and still be usable on older versions of React Native. Ideally, this wouldn't be an issue but with some users having a hard time upgrading it's worth making sure we don't cause too much additional pain, if at all possible.

[fkgozali]

How does this affect current RN libraries? Will it continue to be possible to use the currently released libraries when this feature is implemented? I presume this old interface will be deprecated if so.

The plugin annotation can be considered an "addon" to the actual module implementation. As hinted above in the FAQ section:

Q: Is this a hard blocker for Fabric and TurboModule projects?
No, both projects have their own fallback mechanism to load the modules/components. For the initial rollout, those fallbacks will still be used. But long term, we'd like to unify the registration with this one plugin system, deprecating these fallbacks over time.

This means, as long as the fallback mechanism is still alive in core, old modules will continue to work.

If a library launches with support for the new "plugin" system, will it be possible for this new version to be installed on older versions of React Native which does not use the "plugin" system? Essentially, will a library be able to support both the old and new interface, or will they need to release a version which breaks backwards compatibility?

We're not changing any native code syntax for these modules any time soon, so as long as new modules keep using the recommended syntax, they will continue to work without plugin annotation. In fact, this is how we migrate internal modules to the TurboModule/plugin system, all old syntax in code is still preserved atm. Of course, it's up to module owners if they want to move on to the modern mechanism without using any of the old syntax like RCT_EXPORT_MODULE().

That said, at some point in the distant future, the old mechanism will be deprecated for better maintainability. This won't happen any time soon though, not 2019, maybe 2020.

[ericlewis]

Further, for some use cases, one can get a list of registered plugins as follow:
Does this also setup all the classes? It appears to.

[fkgozali]

Does this also setup all the classes? It appears to.

Invoking the declared plugin function loads the classes, but not necessarily instantiate the instances (that will depend on the callsites).