The TinyTeX-2 bundle on macOS is too big for Github releases now

[yihui]

It has exceeded the 2Gb size limit: https://github.com/rstudio/tinytex/actions/runs/3728064550/jobs/6322748726

We knew it might happen someday and here it comes... It may also happen for Windows and Ubuntu at some point.

The solution is to use split to split the big tgz file into 2Gb pieces. Then installation scripts (R, shell, choco, etc.) need to be adjusted accordingly. The tricky part is to figure out how many parts there are. Perhaps we can write the number into the daily release and read it from there. Then download the parts and extract the whole thing.

[cderv]

Good catch ! It was sooner than we thought...

We could also consider another storage option outside of Github as first discussed when migrating tinytex-releases repo. That could be worth it if the option of splitting the file require us to many hack to maintain in install scripts / function.

For example, I am not sure a splitted installed file would be supported by all our installer tools like Scoop for Windows 🤔
need to look closer - if possible, then probably with some hacks.

The tricky part is to figure out how many parts there are. Perhaps we can write the number into the daily release and read it from there.

It seems a good way. I was thinking of a manifest file with the information otherwise - file that would be ship in the release and we could read. Maybe that is what you meant by "write the number into the daily release" though

[yihui]

Maybe that is what you meant by "write the number into the daily release" though

Yes, I meant something like a manifest file.

That could be worth it if the option of splitting the file require us to many hack to maintain in install scripts / function.

Right. I definitely don't like hacks in several places.

Currently we exceeded the limit by 20Mb. I'll first check what has introduced this extra 20Mb before deciding what to do next.

[yihui]

Just tried the xz format and the size can be cut to 1.6Gb (BTW, bz2 gives 1.9Gb). Changing the archive format means we need to change installation scripts accordingly.

I'll first check if upx could help (https://github.com/upx/upx) and consider other compression methods if not.

[yihui]

TeX Live executables optimized by upx are corrupted at least for macOS, so this is not a good option.

Using the xz format can significantly cut the size. This is the way I have chosen for TinyTeX-2 on macOS and Ubuntu. I think it should be good for a few years.

However, the size of the Windows bundle looks concerning. I feel it may blow up this year, so we need to take a sooner action. One solution is to use 7zip to create a self-extracting archive:

7z a -sfx TinyTeX-2.exe $Env:APPDATA\\TinyTeX

@cderv I'll leave it to you to check the size of the exe file. I think it should be much better than zip. I regret not having thought of using sfx earlier.

Assuming we switch from TinyTeX-2.zip to TinyTeX-2.exe, I think we only need to tweak the installation scripts in three places:

tinytex/tools/install-bin-windows.bat

Line 36 in 142cb3e

powershell -Command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('install.zip', '.'); }"

tinytex/R/install.R

Line 436 in 142cb3e

c('zip', 'tar.gz', 'tgz')[os_index]

tinytex/R/install.R

Line 452 in 142cb3e

extract = if (grepl('[.]zip$', pkg)) unzip else untar