Sources of speedup when copying an array to a zarr array

[aamster]

I have a large image, currently stored as a Jpeg2000 file. It needs to be copied into a stacked zarr array containing many images.

I've noticed that if I convert the image to a zarr array with proper chunking and then copy it into the stacked zarr array, it is significantly faster than just reading the image into memory and copying into the stacked zarr array.

For example,

source = zarr.open('/tmp/source.zarr', mode'w', chunks=(5000, 4000, 3), shape=(30000, 40000, 3), dtype='uint16')
source[:] = np.random.randint(low=0, high=255, size=(30000, 40000, 3), dtype='uint16')

dest = zarr.open('/tmp/dest.zarr', mode='w', chunks=(1, 128, 128, 3), shape=(140, 30000, 40000, 3), dtype='uint16')

 # takes < 2 minutes
%%time
dest[0] = source[:]

compared with:

import glymur

jp2 = glymur.Jp2k('/tmp/source.jp2')
dest = zarr.open('/tmp/dest.zarr', mode='w', chunks=(1, 128, 128, 3), shape=(140, 30000, 40000, 3), dtype='uint16')

# takes 5 minutes
dest[0] = jp2[:]

I know that some of this difference in time must be due to the slower decompression algorithm in jp2. I'm wondering what would be other sources for this speedup.

Thanks!