Skip to content

Commit

Permalink
Bump to libZipSharp 1.0.10
Browse files Browse the repository at this point in the history
Context: dotnet/android-libzipsharp#50

When profiling xamarin-android builds with the Mono profiler, I noticed
`libZipSharp` is one of the biggest allocators of `byte[]`:

    Allocation summary
        Bytes      Count  Average Type name
    251089192      57759     4347 System.Byte[]
        94852584 bytes from:
            Xamarin.Tools.Zip.ZipArchive:Close ()
            (wrapper managed-to-native) Xamarin.Tools.Zip.Native:zip_close (intptr)
            (wrapper native-to-managed) Xamarin.Tools.Zip.ZipArchive:stream_callback (intptr,intptr,ulong,Xamarin.Tools.Zip.SourceCommand)
            Xamarin.Tools.Zip.ZipArchive:stream_callback (intptr,intptr,ulong,Xamarin.Tools.Zip.SourceCommand)
            (wrapper alloc) object:ProfilerAllocVector (intptr,intptr)
            (wrapper managed-to-native) object:__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr)
        25673176 bytes from:
            Xamarin.Android.Tasks.ResolveLibraryProjectImports:Extract (System.Collections.Generic.IDictionary`2<string, Microsoft.Build.Framework.ITaskItem>,System.Collections.Generic.ICollection`1<Microsoft.Build.Framework.ITaskItem>,System.Collections.Generic.ICollection`1<Microsoft.Build.Framework.ITaskItem>,System.Collections.Generic.ICollection`1<Microsoft.Build.Framework.ITaskItem>)
            Xamarin.Android.Tools.Files:ExtractAll (Xamarin.Tools.Zip.ZipArchive,string,System.Action`2<int, int>,System.Func`2<string, string>,System.Func`2<string, bool>)
            Xamarin.Tools.Zip.ZipEntry:Extract (System.IO.Stream)
            Xamarin.Tools.Zip.ZipEntry:DoExtract (intptr,System.IO.Stream,Xamarin.Tools.Zip.EntryExtractEventArgs)
            (wrapper alloc) object:ProfilerAllocVector (intptr,intptr)
            (wrapper managed-to-native) object:__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr)
        3780312 bytes from:
            Xamarin.Tools.Zip.ZipEntry:DoExtract (intptr,System.IO.Stream,Xamarin.Tools.Zip.EntryExtractEventArgs)
            (wrapper managed-to-native) Xamarin.Tools.Zip.Native:zip_fread (intptr,byte[],ulong)
            (wrapper native-to-managed) Xamarin.Tools.Zip.ZipArchive:stream_callback (intptr,intptr,ulong,Xamarin.Tools.Zip.SourceCommand)
            Xamarin.Tools.Zip.ZipArchive:stream_callback (intptr,intptr,ulong,Xamarin.Tools.Zip.SourceCommand)
            (wrapper alloc) object:ProfilerAllocVector (intptr,intptr)
            (wrapper managed-to-native) object:__icall_wrapper_mono_gc_alloc_vector (intptr,intptr,intptr)

This seems like *a lot* of `byte[]` allocations.

To improve this, I added a `<PackageReference/>` to `System.Buffers`
and could make use of `ArrayPool`.

`libZipSharp` will likely still allocate some large `byte[]`, but things
should improve because many will be reused.

The changes appear to have saved ~122,523,744 bytes of allocations:

    Allocation summary
        Bytes      Count  Average Type name
    Before:
    251089192      57759     4347 System.Byte[]
    After:
    128565448      31764     4047 System.Byte[]

I saw build performance improvements for the Xamarin.Forms integration
project, the two tasks heavily using `libZipSharp`:

    Before:
    1881 ms  ResolveLibraryProjectImports               1 calls
    3406 ms  BuildApk                                   1 calls
    After:
    1795 ms  ResolveLibraryProjectImports               1 calls
    3150 ms  BuildApk                                   1 calls

I would guess this saves ~350ms on an initial build. Incremental builds
won't be allocating `byte[]` as heavily, but should see some improvement.

I also need to make sure our installers include `System.Buffers.dll`
alongside everywhere we have a copy of `libZipSharp.dll`.
  • Loading branch information
jonathanpeppers committed Mar 5, 2020
1 parent 1a6fcff commit d5ee61f
Show file tree
Hide file tree
Showing 5 changed files with 16 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Configuration.props
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@
<_TestsProfiledAotName Condition=" '$(AndroidEnableProfiledAot)' == 'true' ">-Profiled</_TestsProfiledAotName>
<_TestsBundleName Condition=" '$(BundleAssemblies)' == 'true' ">-Bundle</_TestsBundleName>
<TestsFlavor>$(_TestsProfiledAotName)$(_TestsAotName)$(_TestsBundleName)</TestsFlavor>
<LibZipSharpVersion>1.0.8</LibZipSharpVersion>
<LibZipSharpVersion>1.0.10</LibZipSharpVersion>
<NuGetApiPackageVersion>5.4.0</NuGetApiPackageVersion>
</PropertyGroup>
<PropertyGroup>
Expand Down
8 changes: 8 additions & 0 deletions Documentation/release-notes/4320.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
### Build and deployment performance

* [GitHub PR 4320](https://github.com/xamarin/xamarin-android/pull/4320):
Update to LibZipSharp 1.0.10 which uses `System.Buffers.ArrayPool`
instead of `byte[]` to save on allocations. This reduced the combined time
for the `ResolveLibraryProjectImports` and `BuildApk` tasks from about 5.3
seconds to about 5.0 seconds for a small test Xamarin.Forms app on an
initial clean build.
1 change: 1 addition & 0 deletions build-tools/check-boot-times/check-boot-times.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="System.CommandLine" Version="2.0.0-beta1.20079.1" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
</ItemGroup>
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
</Project>
2 changes: 2 additions & 0 deletions build-tools/installers/create-installers.targets
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@
<_MSBuildFiles Include="$(MSBuildSrcDir)\apkdiff\libZipSharp.dll.config" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\apkdiff\Newtonsoft.Json.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\apkdiff\Newtonsoft.Json-LICENSE.md" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\apkdiff\System.Buffers.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\aprofutil.exe" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\aprofutil.pdb" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\cil-strip.exe" />
Expand Down Expand Up @@ -196,6 +197,7 @@
<_MSBuildFiles Include="$(MSBuildSrcDir)\manifestmerger.jar" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\protobuf-net.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\SgmlReaderDll.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\System.Buffers.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Aapt.targets" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Aapt2.targets" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\Xamarin.Android.Analysis.targets" />
Expand Down
5 changes: 4 additions & 1 deletion build-tools/scripts/MSBuildReferences.projitems
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,17 @@
<ItemGroup>
<PackageReference Include="Xamarin.Build.AsyncTask" Version="0.3.4" GeneratePathProperty="true" />
<PackageReference Include="Xamarin.LibZipSharp" Version="$(LibZipSharpVersion)" GeneratePathProperty="true" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
</ItemGroup>
<ItemGroup>
<!-- Copy system Microsoft.Build*.dll and dependencies for tests to run against. We can remove this
and rely entirely on NuGet assets when mono/msbuild is merged into microsoft/msbuild. -->
<None Include="$(MSBuildToolsPath)\Microsoft.Build*.dll" Condition=" '$(OS)' == 'Windows_NT' " >
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
<None Include="$(MSBuildToolsPath)\System.*.dll" Condition=" '$(OS)' == 'Windows_NT' " >
<None Include="$(MSBuildToolsPath)\System.*.dll"
Exclude="$(MSBuildToolsPath)\System.Buffers.dll"
Condition=" '$(OS)' == 'Windows_NT' " >
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</None>
</ItemGroup>
Expand Down

0 comments on commit d5ee61f

Please sign in to comment.