Support for Dynamic Feature Modules

[fruitedev]

Hello,

I know there is support for AppBundles which makes things easier, but i couldn't find whether Dynamic Feature Modules is supported or not. Support of Dynamic Feature Modules will be really really helpful for us so please share any plan you have.

Thanks,

[jonathanpeppers]

@fruitedev I remember there might have been some limitations when I first implemented app bundles:

https://github.com/xamarin/xamarin-android/blob/master/Documentation/guides/app-bundles.md#what-are-app-bundles

I remember something about needing to support "Instant Apps" for dynamic modules to work, which has a hard file-size limit. The Mono runtime was over the limit.

We should still look into this again. Anyone interested, please give a thumbs up above and that will help us know the demand, thanks!

[johnthiriet]

@jonathanpeppers Let's consider that the module I want to load dynamically is an aar archive with some jar dependencies and I have a binding over this one. Would that make the process easier on your side ? I mean I do want to load this aar dynamically since it's pretty heavy.

[evansmj]

Now that Apple has introduced App Clips, clients are now more interested in Android Instant Apps. Using app bundles may not be so possible due to the Mono runtime limit and that is fine, but if there was a way we could at least attach natively written instant app apk's as feature modules to the Xamarin.Android app that would be great.

[johnthiriet]

Still really really interested in the dynamic delivery for our applications. We are open for a quick call to explain more about why if you want.
@jonathanpeppers @davidortinau.

[dellis1972]

I found that a community member has already bound the PlayCore library, https://github.com/PatGet/XamarinPlayCoreUpdater. This might have enough support to allow you to deliver a natively written apk as a dynamic feature and keep the main app as a Xamarin App.

[johnthiriet]

@dellis1972 Hum that might be but actually "publishing" this module to the play store will maybe be tricky if even possible as the application is in Xamarin. Even by creating a native application just for packaging this module I think it will not work well.
Have you heard about someone doing it ?

[dellis1972]

@johnthiriet I haven't heard of anyone doing it yet. The developer it seems only used it for in app updates.

[dellis1972]

I did some investigation last week as to what might be needed to support this feature.
Firstly the API's required do use this feature are not currently working in https://github.com/PatGet/XamarinPlayCoreUpdater. When trying to use them you will get a javac error at build time. This is down to the actual binding itself. One of the problems is that the SplitInstallManager uses Listener types which make use of Generics. Our binding story for generics is not completely working at this time. So that will be the first hurdle, get the binding working somehow. This might end up being something like a custom .jar which has a wrapper with a simplified API. We then bind the simplified API rather than use the code google provides. While this is not a great approach, it might be our only option.

The next issue is how do we build a "feature". Some investigation into this shows that it is not going to be straight forward.
The "feature" apk/zip needs to be built against the apk of the application. So we need to build any features after we have built the main application , but before we do the final bundling. The best option is to run a custom target before _PrepareBuildApk which will allow us to build the features, and gather any outputs.

The reason we need to build the features after is the packaged_resources zip file that gets produced by aapt2 as part of our build process MUST be passed as a -I argument to aapt2 when building the "feature"

aapt2 link --allow-reserved-package-id --package-id 0x7e --auto-add-overlay -o $(LevelPackageTemp) -I $(JavaPlatformJarPath) -I $(_PackagedResources) --manifest AndroidManifest.xml -A assets

This is required because the entires in the AndroidManifest.xml we need to include in the "Feature" MUST use resources from the main app for its distribution elements. This is a sample of a working AndoridManfiest.xml for an "Asset Feature"

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
        xmlns:tools="http://schemas.android.com/tools"
        xmlns:dist="http://schemas.android.com/apk/distribution"
        android:versionCode="1"
        android:versionName="1.0"
        package="com.infinitespacestudios.adtest"
        featureSplit="level1"
        android:isFeatureSplit="true">
	<dist:module dist:title="@string/level1" dist:instant="false">
		<dist:delivery>
            <dist:on-demand />
   		</dist:delivery>
		<dist:fusing dist:include="false" />
	</dist:module>
    <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="30" />
	<application android:hasCode="false" tools:replace="android:hasCode" />
</manifest>

Note the @string/level1 resource. That item must be present in the main app strings.xml. So we need to build the main app before we build the feature. Also we cannot build the resources using the --proto-format it seems. We need to build as a normal apk then use the aapt2 convert call to change the format later.

aapt2 convert --output-format proto -o $(LevelPackageProtoTemp) $(LevelPackageTemp)

We then need to manipulate the output file to move things around a bit. By default aapt2 places the compiled AndroidManifest.xml in the root of the zip file. However for aab files it needs to be in a manifest subdirectory within the zip. So we need to fix up the zip. Luckily we already have a task that can do that, so we can probably reuse some code to handle this. Any assemblies we include in the "feature" zip file need to be in the root/assemblies folder as per our normal aab code. At this time the runtime does not support "reloading" our assembly list so we cannot support dynamic delivery of assemblies at this time. However with some additional investigation we might be able to support it.

Once we have this "feature" zip file build we can then pass that to the bundletool via the AndroidAppBundleModules ItemGroup (see here)

The current idea on how an end user might use this is that we allow for an additional piece of meta data on the ProjectReference ItemGroup.

  <ItemGroup>
      <ProjectReference Include="Levels\Level1\Level1.csproj">
         <ReferenceOutputAssembly>false</ReferenceOutputAssembly>
         <AndroidDynamicFeature>true</AndroidDynamicFeature>
      </ProjectReference>  
  </ItemGroup>

In this case ReferenceOutputAssembly HAS to be false otherwise our build system will just include the assembly in the final package. AndroidDynamicFeature is the new item and if 'true' means that during packaging, the feature package will also be produced. The nice thing about using ProjectReference is that the feature will be built as part of a normal build.

At runtime once a user has installed a "feature" they can then use CreatePackageContext to get a new instance of an AssetManager which will contain the new assets.

var ass = CreatePackageContext(PackageName, 0);

For assemblies we need to decide what to do. For java apps they can call StartActivity to kick off an activity assuming the feature contains android activities. The AndroidManifest.xml for the feature would also need to have those activity elements in it so android can pick those up. This might work once we have runtime support to detect new split installs. However we are currently unsure how to do this.

The other option is to use Assembly.Load to load the new "feature" assembly. This might be required if you want to call C# code since there is no reference between the main app and the feature. Reflection will probably be needed to create an instance of a feature class and execute methods. We might be able to hook into the runtime Assembly.Load method do so a search in additional apks/directories if we don't find an assembly in our current mapping list. However this will need us to be able to known "where" the new things are installed.

So we have a number of things we need to implement and investigate before we can fully support this. We will probably have to break this up into a smaller set of projects. The initial goal will be to support dynamic delivery of "assets" the follow that up with dynamic delivery of assemblies later since the runtime changes will not be easy.

UPDATE: So more investigation has turned up some interesting info. Each "Feature" needs to have a different package-id. By
default the package-id is 0x7f for an app. What gradle seems to do is DECREMENT this value for each "feature" that is in the project. So your first "feature" will be 0x7e , you second 0x7d etc. So there is a hard limit on the number of features you can have. This value will need to be passed to aapt2 via the --package-id argument in addition to the --allow-reserved-package-id.

Some thoughts on using ProjectReference. I don't think we can use that now, mostly because we probably need to be able to reference the main app from the feature. To do this we need to pass the feature a path to the main app assembly which needs to be built first. So using a ProjectRefernce will not work since they will be built before the main app. So we will probably need to create a new ItemGroup AndroidDynamicFeature which we can then build after the main app.

<ItemGroup>
     <AndroidDynamicFeature Include="Features/Asset/AssetFeature.csproj" />
</ItemGroup>

This will allow us to pass in not only the main app dll but also all the references which the main app have. This will include things like AndroidX.* and GooglePlayServices. So there should be no need to add those things to the "feature" as PackageReference .

So a new todo items are.

• Make sure all calls to BuildAppBundle pass the AndroidAppBundleModules ItemGroup
• Update Aapt2Link task to support multiple items for the JavaPlatformJarPath or provide a new property to allow us to pass additional -I items.
• Create a new Aapt2Convert task to handle converting from apk to aab formats.
• Update Tasks like BuildApk and BuildBaseAppBundle so that we don't always include runtime native libraries and assemblies. These are not required when building a feature zip.
• Update BuildApkSet to allow users to pass modules [Xamarin.Android.Build.Tasks] Allow users to specify modules for apk sets. #5327
• Update BuildApkSet to allow users to use the --local-testing argument which can be used to test dynamic features without having to upload to google [Xamarin.Android.Build.Tasks] Allow users to specify modules for apk sets. #5327.
• Investigate what might be needed to detect when new split installs are done and refresh our assemblies mapping in the runtime.
• Create a new set of .targets which are specific to dynamic features and are geared to its requirements.
• Investigate the idea of auto generating the AndroidManifest.xml for features as the format is slightly different.
• Create a new template for dotnet new for a dynamic feature?
• See if there is a way to allow debugging support when using aab files. This will be required in order to let users test downloading features etc. This cannot be done using apk files since we must build an aab for any of this to work.
• Add support for --package-id and --allow-reserved-package-id to Aapt2Link.
• Update the Resource.designer.cs generator to take in multiple R.txt files.
• Compile the "Feature" against the assemblies from the main app. This will give it access to all the package and references of the main app (like AndroidX). We only want to pack the "Feature" assembly and nothing else.
• Make sure Java binding code is generated and included in the Feature manifest file
• A whole bunch of other stuff which I have not thought about or has not come up yet.

This is just a brain dump of all the stuff I have found other the last couple of days. There are probably more things we need to do but this is a good starting point. To manage some expectations, this is not going to be a quick process. We have not even mentioned IDE integration or the actual end user experience yet. So this is going to be an on going process. But we are looking at it, which is a start :)

[jonathanpeppers]

Ah, so we should be able to knock this out in an afternoon, right??? 👀

[johnthiriet]

@dellis1972 That is the most interesting thing I have read in weeks :-)

[chc7042]

Hi @dellis1972 and all
i saw the arcicle https://github.com/hinojosachapel/DynamicModules and it working correctly. as i know the android app-bundle has an limitation unlike with WPF dynamic loading. main purpose of android app bundle is seperate some size of binary as bundle for reduce initial time for loading. and it could not be splitted because app bundle should be determined at compile time. so it really different with WPF dynamic bundle. how do you think about this?
and more what i want to know is (i am beginer of WPF(C#)) is there no way to loading module look like https://github.com/hinojosachapel/DynamicModules?

[dellis1972]

I'm working on the initial support for this on #5890 .

It would be nice to get some community feedback on how I plan to implement this. You can read the PR and some initial documentation. All of this is subject to change, but it's a good start. I have something working for AndroidAsset locally but we are still a way off from getting other parts like AndroidResource or even Activities etc.

[Eversor]

Looking forward to this feature, happy to know that a Monogame veteran is doing the job ;) may i help you testing it?

[dellis1972]

@Eversor its still a work in progress I'm afraid. I have had to put it on hold for a while because of other more important tasks on Xamarin.Android. That said I did find some time to put a MonoGame specific example together at https://github.com/infinitespace-studios/MonoGameAndroidAssetPackExample for those who are interested 😄

[dellis1972]

For those of you who are interested in asset packs I have a new PR up at #8631.

We could use some feedback on the approach we plan to take on this if anyone has the time :)

[dellis1972]

Asset Pack support has been merged, should be in the next .NET 9 Previous.

Still trying to figure out how we can even support feature packs :/