Staging fails when URL of external dependencies exceed a certain length

[TimGerlach]

Some of our Java applications are using a bound user-provided-service which triggers one of the built-in frameworks to download a Java agent. The corresponding framework will then download the agent file from the internet depending on the URL given as parameter in the user-provided-service.

In our case the download URL is very long, so we face the issue that the buildpack compilation step fails as the buildpack tries to cache the file and thus the limit of file name length (255 characters) being reached:

[DownloadCache] WARN  Unable to download https://this-url-contains-quite-some-chars.my-cf-landscape.com/e/D7FA9B4E-1BB6-4CCB-ABAC-1589DA51A28D/api/v1/deployment/installer/agent/unix/paas/latest?include=java&bitness=64&Api-Token=hVrw3iAwJWV9zvTcengkx 
                into cache /tmp/cache/final: File name too long @ rb_sysopen - /tmp/cache/final/https%3A%2F%2Fthis-url-contains-quite-some-chars.my-cf-landscape.com%2Fe%2FD7FA9B4E-1BB6-4CCB-ABAC-1589DA51A28De%2Fapi%2Fv1%2Fdeployment%2Finstaller%2Fagent%2Funix%2Fpaas%2Flatest?include=java%26bitness=64%26Api-Token=hVrw3iAwJWV9zvTcengkx.etag

Now we are wondering if this is something that has to be fixed by the buildpack, e.g. by caching files with a different (shorter) name, or if the framework provider should make sure that downloadable artifacts don't exceed a certain URL length (e.g. use http headers instead of URL params).

What's your opinion?

[nebhale]

Wow. I commend you on find this 😅 I’d consider this a bug; the buildpack should never define the URI scheme of the repositories it’s connecting to. There is a simple way to deal with this and that’s to hash the URI to create the cached filename. There’s a nice centralized place to do this, so let me see if I can knock something out and have you give it a test. Are you OK testing a snapshot for me?

[nebhale]

@TimGerlach I'm pretty confident in the change, but when you get a moment, it'd be great if you could test it.

[TimGerlach]

Hi @nebhale, thanks for this remarkably fast support. I just verified that the URL hashing indeed solves the issue. The download of the external dependency works now independent from the length of the URL with the latest buildpack from master.

Great 😃

[nebhale]

Sounds great. Release might be a couple of weeks away (we’re on about a three week cadence), but just keep an eye out as it could be sooner.