Downloading an offline region on Android takes ~3x as long as iOS

[danswick]

Platform: iOS and Android
Mapbox SDK version: Android 5.2.0-beta.2, iOS 3.6

Description of benchmark test

@boundsj and I attempted to download the same offline region using the Android and iOS versions of the SDK. We tested on both wifi (in the Mapbox SF office) and LTE, noting network speeds, download times, and final download sizes along the way.

iOS offline region

    CLLocationCoordinate2D sw = CLLocationCoordinate2DMake(40.759096, -74.0024069);
    CLLocationCoordinate2D ne = CLLocationCoordinate2DMake(40.789221, -73.9398247);
    MGLCoordinateBounds bounds = MGLCoordinateBoundsMake(sw, ne);
...
    maximumZoomLevel = 22
    mapView.zoomLevel = 12.183443853857856

Android offline region

LatLngBounds latLngBounds = new LatLngBounds.Builder()
        .include(new LatLng(40.789221, -73.9398247)) // Northeast
        .include(new LatLng(40.759096, -74.0024069)) // Southwest
        .build();
...
OfflineTilePyramidRegionDefinition definition = new OfflineTilePyramidRegionDefinition(
        mapboxMap.getStyleUrl(),
        latLngBounds,
        12,
        22,
        MainActivity.this.getResources().getDisplayMetrics().density);

We both tested using the latest Mapbox Streets style (mapbox://styles/mapbox/streets-v10).

Test results

Mapbox WiFi

Stats

• Ping: 63ms
• Download: 118 Mbps
• Upload: 32.37 Mbps

🍏 iOS: 51 seconds for 78.9 MB
🤖 Android: 3 minutes, 57 seconds for 76.06 MB

LTE

Stats (iOS)

• Network: AT&T
• Ping: 37ms
• Download: 26.36 Mbps
• Upload: 17.25 Mbps

Stats (Android)

• Network: T-Mobile
• Ping: 100 ms
• Download: 5.7 Mbps
• Upload: 15 Mbps

🍏 iOS: 53 seconds for 78.9 MB
🤖 Android: ~3 minutes, 20 seconds for 76.06 MB

Expected behavior

Download speeds are similar between platforms.

Actual behavior

Download speeds are around 3 times longer on Android when compared to iOS.

Note: while the network speeds were significantly slower on our Android test device, the discrepancy in download times is consistent with those found while testing on the same WiFi network, suggesting that network speed is not the bottleneck.

[friedbunny]

I’d speculate that this has to do with:

1. Slow disk speeds — mobile flash is very slow (and even slower if you’re writing to SD).
2. Conservative database settings that prioritize data safety over speed. See Investigate performance-enhancing database schema changes #6334.

[jfirebaugh]

In my experience, offline downloads are usually bottlenecked on gzip content decoding/re-encoding (#3886).

[zugaldia]

Could it be possible that we're creating multiple HTTPRequest instances, each one creating different OkHttpClient instances, which create different connection pools / thread pools instead of reusing existing ones?

From the docs:

OkHttp performs best when you create a single OkHttpClient instance and reuse it for all of your HTTP calls. This is because each client holds its own connection pool and thread pools. Reusing connections and threads reduces latency and saves memory. Conversely, creating a client for each request wastes resources on idle pools.

cc: @tobrun

[tobrun]

Could it be possible that we're creating multiple HTTPRequest instances, each one creating different OkHttpClient instances, which create different connection pools / thread pools instead of reusing existing ones?

The current HTTPRequest implementation has a static field OkHttpClient. This means that all instances of HttpRequest will share the same OKHttpClient so it should reuse it appropriately.

[zugaldia]

@danswick I assume you ran your tests on a physical device. Could you compare your times by running the same download on an emulator? I just did it on my end (same region, on wi-fi) and I got times closer to iOS on an Android emulator (45 sec) while slightly longer (71 sec) on a device (Pixel XL).

@tobrun Still wondering about OkHttp pool's size. By default, OkHttp has a max of 5 connections per host, while core is setting 20. How're we exposing that limit to OkHttp, shouldn't we be setting setMaxRequestsPerHost(20) on the dispatcher?

[danswick]

@zugaldia ah, sorry for the omission. Yes, I was originally testing on a physical Google Pixel phone. I just tested several times using the emulator (emulated Google Pixel running Android O) and consistently got times around 35 seconds for the same region and over the same wireless network with similar speeds as in the OP.

[tobrun]

@zugaldia great catch on the setMaxRequestsPerHost, could you add that?

[zugaldia]

@danswick thanks for the quick test. The fact that emulator's download times are consistently faster seems to support the idea that we're hitting hardware (CPU, I/O) limitations as indicated by @jfirebaugh and @friedbunny above.

@tobrun #10269 ready for review.