Why is the ISIS3 Anaconda tarball so large?

[jessemapel]

The Anaconda ISIS3 tarball is ~500 MB for linux and ~250 MB for mac.

This is far too large. We need to figure out why.

[SgStapleton]

I modified the build.sh script to include the -DCMAKE_BUILD_TYPE=Release flag, but this did not seem to make a difference.

[jlaura]

So I pulled the linux and OSX tar balls from anaconda. The linux one has (decompressed) 900+MB of documentation and ~60MB of binaries. The OSX version has ~90MB of binaries, but 'only' 735MB of docs. So, the problem here is that we are shipping documentation inside of a source code archive.

The fix is going to be to get the documentation out of the upload to anaconda and then push the documentation to someplace more sensical. Also, it looks like the largest files are in the Developer and Programmer docs...

[jlaura]

It also looks like we are shipping user documentation in here on the order of 300MB. This is duplicated between tabbed and printer friendly.

[jessemapel]

For right now, I will probably remove the docs from the build. We can put the docs on rsync or something if we really want people to be able to download them.

[jlaura]

👍 I also heard back from the anaconda team - they do not offer larger storage hosting options.

[thareUSGS]

Do docs need to be on system? Maybe offer online docs, a nicely compressed PDF, or an optional download.

Personally I never use the local docs with ISIS3, just the online resources.

[rbeyer]

Having local documentation is useful to a user when they are not online, and in the past, when the USGS website has had connection problems, having the docs locally was helpful. If the 'online' docs can be hosted on a server with more reliable uptime, then this second concern is mitigated, but you always want to be able to serve the first concern.

So I have some opinions (surprise!):

• hundreds of MB of documentation? But I've seen the ISIS documentation, that's just ... wow. Yeah, that's too much.

• I think the right solution is to figure out how to create a version of the documentation that doesn't have such a large disk footprint. Creating a compressed PDF like Trent suggested is one approach, but even that is doing some duplication, since the docs exist as XML. Maybe provide a command in the isis conda download that allows the user to 'build' the documentation locally from the XML that they already have (rather than doing it for them ahead of time, and then shipping duplicate info)? Or include a standard command in ISIS programs, such that "--man" or something grabs the XML documentation and spits it out to the screen.

• If you can't make it smaller, then separating it as a separate download or conda package is the route that you should take until you can shrink it, for those users that need it.

[jessemapel]

@rbeyer I like the idea of having a command to build the app docs in the release. We cannot build the dev docs in the release though. A lot of the documentation is in the cpp files that are not included in the distribution.

[rbeyer]

I think that would be fine. I think the 'need' of offline docs is the app-level docs that give more info than what you get from -h.

[jlaura]

I just created an RFC for this so that we can have a clear idea of what is being discussed. We can use this issue to get everyone on the same page and then go ahead and get PR(s) in with the necessary changes.

[jessemapel]

For reference, I am removing the docs from the current release because it shaves several hours off of the packaging and build time. This has caused the OSX tarball to go from 200 MB to 12MB.

[jessemapel]

No one has provided any more feedback against this, so I'm going to go ahead and close this out ~2 weeks from now on February 28th.

[lwellerastro]

Are you all talking about the online documentation? I'm trying to understand what the impact might be locally, but I don't understand the problem well enough to ask the right question or make an appropriate comment. I have the ISIS 3 Application Documentation web page opened all of the time as work - is that going to be impacted somehow?

[jessemapel]

@lwellerastro The online docs will not be impacted. We previously provided an offline copy of the docs with every ISIS install. Because it takes up about 50 times as much space as the rest of the ISIS install, it's the logical place to reduce the size of our install.

[lwellerastro]

Thanks @jessemapel !

[jessemapel]

@ALL Still planning to close this tomorrow unless someone thinks we shouldn't. So, far this has de facto been implemented for 3.6.1 and 3.6.2 due to cloud size limitations.

[rbeyer]

Is there a fix for those individuals that want an offline version of the docs, or a version hosted elsewhere? For example, during the government shutdown the ISIS documentation website was unavailable.

[jessemapel]

You are still able to build the documentation from the repository. After configuring with cmake the docs target will build all of the documentation locally. We can look into app specific documentation once we have time.

[rbeyer]

While not ideal, it is certainly sufficient. Seems good to close.

[jessemapel]

Closing this off:

• We're not going to be distributing docs with the binaries
• The full documentation can be built via the docs target
• Something that would be convenient for the future is a tool to just build simple app documentation