spec & prototype of Android App Bundles

Context: #2727

This is a prototype that gets us this far:

* We generate a `.aab` file
* We can generate a `.apks` file specific for an attached device
* We can *install* the `.apks` file
* The app starts successfully!

This workflow is achieved by:

* The `<Aapt2Link/>` MSBuild task needs to pass `--proto-format`.
* The `<AppBundleBaseZip/>` and `<BundleToolBuildBundle/>` MSBuild
  tasks run instead of `<BuildApk/>`.
* The `<BundleToolBuildApkSet/>` and `<BundleToolInstallApkSet/>`
  tasks run instead of `adb install`. These are somewhat odd, but they
  use the attached device to decide which format APK set is needed.
  Otherwise the APK set was 200MB!

Some notes about Android App Bundles:

* App bundles use `android:extractNativeLibs="false"`, unless the
  target device's API level is too low.
* `$(AndroidUseAapt2)` is required.
* `$(EmbedAssembliesIntoApk)` is required.
* `$(_EmbeddedDSOsEnabled)` is required, regardless of
  `android:extractNativeLibs` value in `AndroidManifest.xml`.
* `$(AndroidUseApkSigner)` is turned off.
* `$(AndroidUseSharedRuntime)` is turned off.

~~ Java.Interop ~~

Some changes are needed in `MonoRuntimeProvider.Bundled.java` in
java.interop before we can merge this.

We need the "split apks" to be in the list of APKs by calling
`ApplicationInfo.splitPublicSourceDirs`:

https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#splitPublicSourceDirs

Documentation/guides/app-bundles.md

@@ -0,0 +1,287 @@
+This is the Android App Bundle and `bundletool` integration
+specification for Xamarin.Android.
+
+# What are "app bundles"?
+
+[Android App Bundles][app_bundle] are a new publishing format for
+Google Play that has a wide array of benefits.
+
+* You no longer have to upload multiple APKs to Google Play:
+
+> With the Android App Bundle, you build one artifact that includes
+> all of your app's compiled code, resources, and native libraries for
+> your app. You no longer need to build, sign, upload, and manage
+> version codes for multiple APKs.
+
+* "Dynamic Delivery" provides an optimized APK download from Google
+  Play:
+
+> Google Play’s Dynamic Delivery uses your Android App Bundle to build
+> and serve APKs that are optimized for each device configuration.
+> This results in a smaller app download for end-users by removing
+> unused code and resources needed for other devices.
+
+These first two features of Android App Bundles are a natural fit for
+Xamarin.Android apps. The first version of `bundletool` support in
+Xamarin.Android will focus on these two benefits.
+
+*Unfortunately* the next two features will be more involved. We could
+perhaps support them in Xamarin.Android down the road.
+
+* Support for "Instant Apps":
+
+> Instant-enable your Android App Bundle, so that users can launch an
+> instant app entry point module from the Try Now button on Google
+> Play and web links without installation.
+
+Xamarin.Android does not yet have full support for [Instant
+Apps][instant_apps], in general. There would likely be some changes
+needed to the runtime, and there is a file size limit on the base APK
+size. App Bundles won't necessarily help anything for this.
+
+* Deliver features on-demand:
+
+> Further reduce the size of your app by installing only the features
+> that the majority of your audience use. Users can download and
+> install dynamic features when they’re needed. Use Android Studio 3.2
+> to build apps with dynamic features, and join the beta program to
+> publish them on Google Play.
+
+For Xamarin.Android to implement this feature, I believe Instant App
+support is needed first.
+
+For more information on App Bundles, visit the [getting
+started][getting_started] guide.
+
+[app_bundle]: https://developer.android.com/platform/technology/app-bundle
+[instant_apps]: https://developer.android.com/topic/google-play-instant
+[getting_started]: https://developer.android.com/guide/app-bundle/
+
+# What is `bundletool`?
+
+[bundletool][bundletool] is the underlying command-line tool that
+gradle, Android Studio, and Google Play use for working with Android
+App Bundles.
+
+Xamarin.Android will need to run `bundletool` for the following cases:
+
+* Create an Android App Bundle from a "base" zip file
+* Create an APK Set (`.apks` file) from an Android App Bundle
+* Deploy an APK Set (`.apks` file) to a device or emulator
+
+The help text for `bundletool` reads:
+
+```
+Synopsis: bundletool <command> ...
+
+Use 'bundletool help <command>' to learn more about the given command.
+
+build-bundle command:
+    Builds an Android App Bundle from a set of Bundle modules provided as zip
+    files.
+
+build-apks command:
+    Generates an APK Set archive containing either all possible split APKs and
+    standalone APKs or APKs optimized for the connected device (see connected-
+    device flag).
+
+extract-apks command:
+    Extracts from an APK Set the APKs that should be installed on a given
+    device.
+
+get-device-spec command:
+    Writes out a JSON file containing the device specifications (i.e. features
+    and properties) of the connected Android device.
+
+install-apks command:
+    Installs APKs extracted from an APK Set to a connected device. Replaces
+    already installed package.
+
+validate command:
+    Verifies the given Android App Bundle is valid and prints out information
+    about it.
+
+dump command:
+    Prints files or extract values from the bundle in a human-readable form.
+
+get-size command:
+    Computes the min and max download sizes of APKs served to different devices
+    configurations from an APK Set.
+
+version command:
+    Prints the version of BundleTool.
+```
+
+The source code for `bundletool` is on [Github][github]!
+
+[bundletool]: https://developer.android.com/studio/command-line/bundletool
+[github]: https://github.com/google/bundletool
+
+# Implementation
+
+To enable app bundles, a new MSBuild property is needed:
+
+```xml
+<AndroidPackageFormat>bundletool</AndroidPackageFormat>
+```
+
+`$(AndroidPackageFormat)` will default to `apk` for the current
+Xamarin.Android behavior. Since it impacts deployment, using
+`bundletool` will also need to turn off Xamarin.Android's "Fast
+Deployment" feature.
+
+Due to the various requirements for Android App Bundles, here are a
+reasonable set of defaults for `bundletool`:
+```xml
+<AndroidPackageFormat    Condition=" '$(AndroidPackageFormat)' == '' ">apk</AndroidPackageFormat>
+<AndroidUseAapt2         Condition=" '$(AndroidPackageFormat)' == 'bundletool' ">True</AndroidUseAapt2>
+<AndroidUseApkSigner     Condition=" '$(AndroidPackageFormat)' == 'bundletool' ">False</AndroidUseApkSigner>
+<EmbedAssembliesIntoApk  Condition=" '$(AndroidPackageFormat)' == 'bundletool' ">True</EmbedAssembliesIntoApk>
+<AndroidUseSharedRuntime Condition=" '$(AndroidPackageFormat)' == 'bundletool' ">False</AndroidUseSharedRuntime>
+```
+
+Adding `<AndroidPackageFormat>` most commonly be done for `Release`
+builds for submission to Google Play:
+
+```xml
+<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
+  <!--...-->
+  <AndroidPackageFormat>bundletool</AndroidPackageFormat>
+</PropertyGroup>
+```
+
+Using `$(AndroidPackageFormat)` for `Debug` builds would impact the
+dev-loop dramatically, since it disables "Fast Deployment". It also
+takes some time for `bundletool` to generate an app bundle and
+device-specific APK set to be deployed.
+
+## aapt2
+
+The first requirement is that App Bundles require a special protobuf
+format for resource files that can only be produced by `aapt2`. Adding
+the `--proto-format` flag to the `aapt2` call produces a
+`resources.pb` file:
+
+```
+aapt2 link [options] -o arg --manifest arg files...
+
+Options:
+...
+
+--proto-format
+
+Generates compiled resources in Protobuf format.
+Suitable as input to the bundle tool for generating an App Bundle.
+```
+
+This command-line switch is new in `aapt2` and can only be used with
+the version of `aapt2` from Maven. We are now shipping this new
+version in Xamarin.Android.
+
+## Generate a base ZIP file
+
+Once we have a `resources.pb` file, we must generate a [base ZIP
+file][zip_format] of the following structure:
+
+* `manifest/AndroidManifest.xml`: in protobuf format
+* `dex/`: all `.dex` files
+* `res/`: all Android resources
+* `assets/`: all Android assets
+* `lib/`: all native libraries (`.so` files)
+* `root/`: any arbitrary files that need to go in the root of the
+  final APK on-device. Xamarin.Android will need to put .NET
+  assemblies in `root/assemblies`.
+* `resources.pb`: the resource table in protobuf format
+
+See the [.aab format spec][aab_format] for further detail.
+
+[zip_format]: https://developer.android.com/studio/build/building-cmdline#package_pre-compiled_code_and_resources
+[aab_format]: https://developer.android.com/guide/app-bundle#aab_format
+
+## BundleConfig.json
+
+Since .NET assemblies and typemap files must remain uncompressed in
+Xamarin.Android apps, we will also need to specify a
+`BundleConfig.json` file:
+
+```json
+{
+  "compression": {
+    "uncompressedGlob": ["typemap.mj", "typemap.jm", "assemblies/*"]
+  }
+}
+```
+
+We also must include rules for what is specified in
+`$(AndroidStoreUncompressedFileExtensions)`, which is currently a
+delimited list of file extensions. Prepending `**/*` to each extension
+should match the glob-pattern syntax that `bundletool` expects.
+
+See details about `BundleConfig.json` in the [app bundle
+documentation][bundleconfig_json], or the [proto3 declaration on
+Github][bundleconfig_proto].
+
+From here we can generate a `.aab` file with:
+
+```
+bundletool build-bundle --modules=base.zip --output=foo.aab --config=BundleConfig.json
+```
+
+[bundleconfig_json]: https://developer.android.com/studio/build/building-cmdline#bundleconfig
+[bundleconfig_proto]: https://github.com/google/bundletool/blob/8e3aef8dd8ba239874008df33324b6f343261139/src/main/proto/config.proto
+
+## Native Libraries
+
+It appears that app bundles use `android:extractNativeLibs="false"` by
+default, so that native libraries remain in the APK, but stored
+uncompressed.
+
+They take it even further, in that the current default behavior
+(`extractNativeLibs="true"`) cannot be enabled, and is only enabled on
+older API levels:
+
+    // Only the split APKs targeting devices below Android M should be compressed. Instant apps
+    // always support uncompressed native libraries (even on Android L), because they are not always
+    // executed by the Android platform.
+
+This means that developer's `extractNativeLibs` setting in their
+`AndroidManifest.xml` is basically ignored. See [bundletool's source
+code][nativelibs] for details.
+
+[nativelibs]: https://github.com/google/bundletool/blob/fe1129820cb263b3fef18ab7e95d80c228c065a1/src/main/java/com/android/tools/build/bundletool/splitters/NativeLibrariesCompressionSplitter.java#L74-L78
+
+## Signing
+
+App Bundles can only be signed with `jarsigner` (not `apksigner`). App
+Bundles do not need to use `zipalign`. Xamarin.Android should go ahead
+and sign the `.aab` file the same as it currently does for `.apk`
+files. A `com.company.app-Signed.aab` file will be generated in
+`$(OutputPath)`, to match our current behavior with APK files.
+
+Google Play has recently added support for [doing the final,
+production signing][app_signing], but Xamarin.Android should sign App
+Bundles with what is configured in the existing MSBuild properties.
+
+[app_signing]: https://developer.android.com/studio/publish/app-signing
+
+## Deployment
+
+First, we will need to invoke `bundletool` to create an APK set:
+
+```
+bundletool build-apks --bundle=foo.aab --output=foo.apks
+```
+
+Running the [build-apks][build_apks] command, generates a `.apks` file.
+
+To deploy a `.apks` file to a connected device:
+
+```
+bundletool install-apks --apks=foo.apks
+```
+
+The [install-apks][install_apks] command will *finally* get the app
+onto the device!
+
+[build_apks]: https://developer.android.com/studio/command-line/bundletool#generate_apks
+[install_apks]: https://developer.android.com/studio/command-line/bundletool#deploy_with_bundletool

src/Xamarin.Android.Build.Tasks/Tasks/Aapt2Link.cs

@@ -71,6 +71,8 @@ public class Aapt2Link : Aapt2 {
 
 		public bool NonConstantId { get; set; }
 
+		public bool ProtobufFormat { get; set; }
+
 		AssemblyIdentityMap assemblyMap = new AssemblyIdentityMap ();
 		List<string> tempFiles = new List<string> ();
 
@@ -174,6 +176,9 @@ string GenerateCommandLineCommands (string ManifestFile, string currentAbi, stri
 			if (!string.IsNullOrEmpty (ResourceSymbolsTextFile))
 				cmd.AppendSwitchIfNotNull ("--output-text-symbols ", ResourceSymbolsTextFile);
 
+			if (ProtobufFormat)
+				cmd.AppendSwitch ("--proto-format");
+
 			var extraArgsExpanded = ExpandString (ExtraArgs);
 			if (extraArgsExpanded != ExtraArgs)
 				Log.LogDebugMessage ("  ExtraArgs expanded: {0}", extraArgsExpanded);

src/Xamarin.Android.Build.Tasks/Tasks/AndroidSignPackage.cs

@@ -34,10 +34,14 @@ public class AndroidSignPackage : AndroidToolTask
 
 		public string SigningAlgorithm { get; set; }
 
+		public string FileSuffix { get; set; }
+
 		protected override string DefaultErrorCode => "ANDJS0000";
 
 		protected override string GenerateCommandLineCommands ()
 		{
+			var fileName = Path.GetFileNameWithoutExtension (UnsignedApk);
+			var extension = Path.GetExtension (UnsignedApk);
 			var cmd = new CommandLineBuilder ();
 
 			cmd.AppendSwitchIfNotNull ("-tsa ", TimestampAuthorityUrl);
@@ -47,7 +51,7 @@ protected override string GenerateCommandLineCommands ()
 			cmd.AppendSwitchIfNotNull ("-keypass ", KeyPass);
 			cmd.AppendSwitchIfNotNull ("-digestalg ", "SHA1");
 			cmd.AppendSwitchIfNotNull ("-sigalg ", string.IsNullOrWhiteSpace (SigningAlgorithm) ? "md5withRSA" : SigningAlgorithm);
-			cmd.AppendSwitchIfNotNull ("-signedjar ", String.Format ("{0}{1}{2}-Signed-Unaligned.apk", SignedApkDirectory, Path.DirectorySeparatorChar, Path.GetFileNameWithoutExtension (UnsignedApk)));
+			cmd.AppendSwitchIfNotNull ("-signedjar ", Path.Combine (SignedApkDirectory, $"{fileName}{FileSuffix}{extension}" ));
 
 			cmd.AppendFileNameIfNotNull (UnsignedApk);
 			cmd.AppendSwitch (KeyAlias);

src/Xamarin.Android.Build.Tasks/Tasks/AppBundleBaseZip.cs

@@ -0,0 +1,43 @@
+using Microsoft.Build.Framework;
+using Microsoft.Build.Utilities;
+using System;
+using System.IO;
+using Xamarin.Tools.Zip;
+
+namespace Xamarin.Android.Tasks
+{
+	public class AppBundleBaseZip : BuildApk
+	{
+		/// <summary>
+		/// Files that need to land in the final APK need to go in `root/`
+		/// </summary>
+		protected override string RootPath => "root/";
+
+		/// <summary>
+		/// `.dex` files should be in `dex/`
+		/// </summary>
+		protected override string DalvikPath => "dex/";
+
+		/// <summary>
+		/// Nothing needs to be compressed with app bundles. BundleConfig.json specifies the final compression mode.
+		/// </summary>
+		protected override CompressionMethod UncompressedMethod => CompressionMethod.Default;
+
+		/// <summary>
+		/// aapt2 is putting AndroidManifest.xml in the root of the archive instead of at manifest/AndroidManifest.xml that bundletool expects.
+		/// I see no way to change this behavior, so we can move the file for now:
+		/// https://github.com/aosp-mirror/platform_frameworks_base/blob/e80b45506501815061b079dcb10bf87443bd385d/tools/aapt2/LoadedApk.h#L34
+		/// </summary>
+		protected override void FixupArchive (ZipArchiveEx zip)
+		{
+			var entry = zip.Archive.ReadEntry ("AndroidManifest.xml");
+			using (var stream = new MemoryStream ()) {
+				entry.Extract (stream);
+				stream.Position = 0;
+				zip.Archive.AddEntry ("manifest/AndroidManifest.xml", stream);
+				zip.Archive.DeleteEntry (entry);
+				zip.Flush ();
+			}
+		}
+	}
+}